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IDENTIFYING  CRITICAL  MANNED-UNMANNED  TEAMING  SKILLS  FOR  UNMANNED 
AIRCRAFT  SYSTEM  OPERATORS 

EXECUTIVE  SUMMARY 


Research  Requirements: 

In  2003,  the  mission  of  developing  and  employing  United  States  (US)  Army  unmanned 
aircraft  systems  (UAS)  passed  from  Military  Intelligence  to  the  Aviation  branch.  The  focus  of 
the  UAS  mission  changed  from  strategic  intelligence,  surveillance,  and  reconnaissance  to  scout- 
reconnaissance  (SR)  operations.  The  shift  has  produced  an  increased  requirement  for 
coordination  between  manned  and  unmanned  aircraft.  The  tactics,  techniques,  and  procedures 
for  manned-unmanned  teaming  (MUM-T)  are  still  evolving,  and  there  is  as  yet  no  common 
understanding  of  how  manned  and  unmanned  aircraft  will  interoperate  to  perform  SR  missions. 
Thus,  there  is  a  need  for  understanding  (a)  the  specific  skills  required  for  such  tasks  and  (b)  the 
most  efficient  ways  to  train  those  skills. 

The  overall  objective  of  this  research  is  to  specify  an  empirically-derived  set  of  MUM-T 
critical  skills  and  benchmark  indicators  for  performance  on  those  skills.  Specifically,  the 
research  is  designed  to  (a)  identify  the  MUM-T  coordination  skills  required  for  UAS  aircrews, 
(b)  define  training-critical  MUM-T  aircrew  skills,  and  (c)  determine  benchmark  performance 
indicators  of  training-critical  skills  that  can  be  used  to  develop  training  metrics.  The  project 
concentrates  on  the  RQ-7B  Shadow,  which  is  the  most  numerous  UAS  currently  in  service  with 
the  Army. 

Procedure: 

The  research  began  with  a  review  of  Army  doctrinal  material  and  regulations,  other 
related  published  materials,  and  other  sources  to  identify  (a)  the  missions  in  which  UAS 
operators  would  need  to  coordinate  with  helicopter  pilots,  (b)  the  specific  tasks  required  to 
perform  these  missions  in  which  manned  and  unmanned  systems  would  interact,  and  (c)  the 
MUM-T  skills  involved  in  conducting  these  tasks.  The  results  of  this  review,  a  list  of  training- 
critical  skills,  were  confirmed  as  relevant  to  MUM-T  and  doctrinally  correct  in  a  workshop 
attended  by  UAS  operators,  helicopter  pilots,  and  doctrine  developers. 

An  additional  set  of  workshops  was  then  conducted  to  establish  the  training  criticality  of 
these  skills.  Training  critical  skills  were  identified  on  the  basis  of  two  criteria:  (a)  inadequate 
performance  would  likely  lead  to  mission  failure  or  would  present  a  serious  risk  to  personnel  or 
equipment,  and  (b)  UAS  operators  recently  graduated  from  Advanced  Individual  Training  (AIT) 
do  not  possess  these  skills.  Ratings  of  these  skills  by  workshop  participants  (UAS  operators  and 
helicopter  pilots)  were  used  to  establish  an  ordering  of  the  skills  by  training  criticality. 

A  third  set  of  workshops  elicited  descriptions  of  performance  indicators  from  a  third, 
similar  group  of  experienced  individuals  representing  both  the  manned  and  UAS  operator 
communities.  These  behavioral  indicators  signal  competency  (or  non-competency)  in  the  skills 
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previously  identified  as  being  relevant  to  MUM-T.  The  workshop  also  defined  the  conditions 
under  which  these  indicators  would  be  likely  to  occur  and  be  observable. 

Findings: 

A  total  of  25  MUM-T  skills  were  initially  identified.  The  average  criticality  ratings 
indicated  that  all  the  skills  were  judged  to  be  at  least  moderately  important  to  attack  and 
reconnaissance  missions  and  to  present  moderate  to  serious  risks  to  personnel  and  equipment  if 
performed  incorrectly.  Performance  levels  (i.e.,  perceived  levels  of  competency  required  for  a 
given  skill)  varied  greatly,  which  indicated  that  some  skills  were  not  addressed  in  AIT  including 
skills  important  to  reconnaissance  or  attack  missions.  Others,  though  important  were  found  to  be 
trained  to  a  relatively  high  level  of  competence.  A  total  of  20  MUM-T  skills  were  determined  to 
be  training-critical  skills. 

Indicators,  conditions  under  which  performance  was  likely  to  be  observable  and  the 
environments  in  which  they  might  be  observed  (i.e.,  sources)  were  identified  for  the  20  training- 
critical  skills  that  were  deemed  most  relevant  to  MUM-T.  Overall,  140  behavioral  indicators 
were  identified  during  the  workshops.  In  addition,  there  were  41  responses  for  conditions  and  80 
responses  for  sources.  The  available  training  delivery  environments  should  be  reviewed  to 
determine  more  specifically  the  source  where  MUM-T  performance  indicators  may  be 
observable.  Each  of  the  training  delivery  environments  should  be  analyzed  for  content  (e.g. 
scenarios)  that  contain  the  conditions  under  which  the  indicators  may  be  observable. 

Utilization  and  Dissemination  of  Findings: 

The  results  of  this  research  can  be  used  to  identify  training  needs,  select  training  methods, 
and  develop  ways  to  assess  performance  on  MUM-T  skills.  In  particular,  the  training  criticality 
assessments  describe  the  critical  MUM-T  skills  that  are  not  currently  being  addressed  in  AIT.  In 
addition,  further  analysis  of  the  indicators,  conditions,  and  sources  should  lead  to  the 
development  of  automated  and  observational  approaches  to  performance  assessment.  Results  of 
this  research  effort  were  briefed  to  the  U.S.  Anny  Aviation  TRADOC  Capability  Managers  for 
Reconnaissance- Attack  and  for  UAS  on  10  February  2012. 
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Identifying  Critical  Manned-Unmanned  Teaming  Skills 
for  Unmanned  Aircraft  System  Operators 

Introduction 

In  2002,  there  were,  in  all  forces  (Army,  Air  Force,  Navy,  and  Marine  Corps),  a  total  of 
167  unmanned  aircraft  of  all  types.  Today,  the  Army  alone  has  over  4,000  unmanned  aircraft  and 
has  amassed  more  than  one  million  operational  flight  hours  (United  States  Army  UAS  Center  of 
Excellence,  2010).  In  2003,  the  mission  of  developing  and  employing  unmanned  aircraft 
systems  (UAS)  passed  from  the  Military  Intelligence  (MI)  branch  to  the  Aviation  branch. 

During  this  transition,  the  UAS  mission  has  changed  from  strategic  intelligence,  surveillance, 
and  reconnaissance  (ISR)  to  tactical  scout-reconnaissance  (SR).  The  shift  in  mission  was  made  in 
recognition  of  the  need  to  integrate  UAS  more  centrally  into  aviation  operations.  As  a  result, 
there  is  a  growing  need  for  manned-unmanned  teaming  (MUM-T)  during  aviation  SR  missions. 

As  MUM-T  coordination  becomes  more  critical  to  the  missions  conducted  by  UAS, 
MUM-T  should  have  correspondingly  greater  emphasis  in  UAS  operator  training.  However, 
recent  research  indicated  that  initial  training  of  RQ-7B  Shadow  operators  often  lacked  specific 
skills  required  to  execute  tactical  aviation  SR  missions  (Stewart,  Bink,  Barker,  Tremlett,  &  Price, 
2011).  The  lack  of  formal  SR  skills  training  makes  UAS  operators  less  prepared  to  interact  with 
manned  aircrews.  Moreover,  MUM-T  doctrine  currently  is  in  its  infancy  and  still  evolving.  The 
lack  of  specific  MUM-T  training,  plus  still-emerging  MUM-T  doctrine  lead  to  a  level  of 
uncertainty  in  how  manned  and  unmanned  aircraft  will  interoperate  to  perform  SR  missions. 

The  respective  roles  of  manned  aircraft  and  UAS  in  MUM-T  are  also  unclear.  For  example,  a 
recent  survey  of  Army  helicopter  pilots  and  UAS  operators  revealed  substantial  differences 
between  the  two  groups  regarding  the  perceived  capabilities  and  roles  of  manned  aircraft  and 
UAS  in  the  execution  of  aviation  SR  missions  (Stewart,  Roberts  &  Bink,  2012).  Because  MUM- 
T  skills  are  relatively  new  to  both  helicopter  aircrews  and  UAS  operators,  it  is  crucial  to 
understand  (a)  specific  skills  required  for  MUM-T  task  performance  and  (b)  the  most  efficient 
ways  to  train  UAS  operators  on  MUM-T  skills. 

The  concept  of  MUM-T  implies  that  the  interaction  between  manned  aircraft  and  UAS 
capitalizes  on  the  strengths  of  the  two  types  of  aircraft.  For  example,  the  tactics,  techniques,  and 
procedures  (TTP)  for  MUM-T  have  emerged  in  part  through  the  maturation  of  Task  Force  ODIN 
(Observe,  Detect,  Identify,  Neutralize),  which  was  a  counter  improvised  explosive  device 
program  started  in  2006  (U.S.  Army  UAS  Center  of  Excellence,  2010).  At  the  core  of  these  TTP 
was  the  creation  of  linkages  between  sensors  and  shooters  wherein  UAS  sensor  operators  acquire 
information  to  develop  the  enemy  situation  and  the  shooters  (i.e.,  armed  helicopters)  are  able  to 
attack  the  target  at  safer-and  more  covert-  standoff  distances  (sometimes  without  even  having  to 
see  the  target  itself).  In  addition  MUM-T  TTPs  have  been  developed  from  simulation-based 
experimentation  (Air  Maneuver  Battle  Lab,  1998,  1999a,  1999b,  2000;  Howse  &  Cross,  1998). 
The  evolution  of  MUM-T  TTP  has  been  further  evidenced  by  the  development  of  new 
organizational  structures  in  which  UAS  will  operate.  Currently,  UAS  Platoons  are  incorporated 
as  Brigade  level  assets  in  either  light  or  heavy  Brigade  Combat  Teams  (BCT),  which  are  ground 
units.  However,  the  newly  organized  Full  Spectrum  Combat  Aviation  Brigade  (FSCAB)  will 
incorporate  organic  UAS,  including  an  Air  Cavalry  Squadron  in  which  an  eight-aircraft  RQ-7B 
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Troop  will  replace  nine  of  the  Squadron’s  OH-58D  Kiowa  Warrior  anned  SR  helicopters.  The 
FSCAB  will  also  be  capable  of  performing  MUM-T  operations  with  Division-level  MQ-1C  Gray 
Eagle  heavy  UAS. 

Altogether,  there  are  indications  of  what  types  of  tactical  skills  should  be  involved  in 
MUM-T,  but  the  specifics  of  teaming  skills  are  yet  to  be  defined.  There  are  particular  team-level 
skills  required  of  both  manned  and  UAS  aircrews  in  order  to  successfully  execute  MUM-T 
missions.  For  example,  attacking  an  identified  target  could  require  that  the  UAS  aircrew  identify 
the  target,  call  for  fire  support  from  an  AH-64D,  laser-designate  the  target  and  hand  the  target  off 
to  the  AH-64D.  How  those  specific  MUM-T  skills  are  defined  and  classified  will  impact  future 
training  for  UAS  operators.  More  importantly,  perfonnance  measures  of  MUM-T  skills  are 
needed  both  to  ensure  the  effectiveness  of  MUM-T  training  and  to  aid  in  further  defining 
appropriate  MUM-T  skills  (e.g.,  Cannon-Bowers  &  Salas,  1997). 

Technical  Objectives 

The  goal  of  this  research  effort,  undertaken  by  the  United  States  (US)  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences  (ARI)  was  to  identify  an  empirically-derived  set 
of  MUM-T  critical  skills  and  to  develop  benchmark  indicators  of  performance  on  those  skills. 
Specifically,  the  research  was  designed  to  meet  the  following  objectives: 

1.  Identify  the  MUM-T  coordination  skills  required  for  UAS  aircrews. 

2.  Define  training-critical  MUM-T  aircrew  skills. 

3.  Determine  benchmark  indicators  of  MUM-T  training-critical  skills  that  can  be  used  to 

develop  training  metrics. 

The  research  concentrated  on  the  most  numerous  UAS  currently  in  service  with  the 
Army,  the  RQ-7B  Shadow.  RQ-7B  currently  is  a  BCT  asset  and  associated  with  a  Product 
Improvement  Program  that  should  allow  for  its  employment  in  tactical  UAS  operations  through 
the  end  of  2015.  However,  the  ongoing  stand-up  of  FSCABs  will  likely  increase  the  RQ-7B’s 
operational  longevity  beyond  that  date. 

Overview  of  Technical  Approach 

The  current  research  project  began  with  a  review  of  Anny  doctrinal  material  and 
regulations,  other  Army  documents,  and  other  published  materials  to  identify  (a)  the  missions  in 
which  UAS  operators  would  need  to  coordinate  with  helicopter  pilots,  (b)  the  specific  tasks 
required  to  perform  these  missions  in  which  manned  and  unmanned  systems  would  interact,  and 
(c)  the  MUM-T  skills  involved  in  conducting  these  tasks.  Following  the  initial  review,  three 
types  of  workshops  were  convened.  In  general,  the  3-4  hour  workshops  elicited  the  knowledge 
of  subject-matter  experts  (SME)  regarding  MUM-T  skills  required  for  successful  completion  of 
reconnaissance  and  attack  missions.  Workshop  SMEs  included  including  UAS  operators, 
helicopter  pilots,  and  doctrine  developers.  The  current  methodology  has  some  similarities  to  the 
Air  Force  Research  Laboratory’s  Mission  Essential  Competencies  (MEC)  approach  (Colegrove 
&  Bennett,  2006),  but  the  current  approach  is  not  a  derivative  of  the  Air  Force  approach. 
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The  first  set  of  workshops  reviewed  and  confirmed  the  tasks  and  skills  that  were 
identified  in  published  documentation.  A  second  set  of  workshops  was  then  conducted  to 
establish  the  training  criticality  of  the  identified  skills.  Training-critical  skills  were  identified  on 
the  basis  of  two  criteria:  (a)  inadequate  performance  of  the  skills  would  likely  lead  to  mission 
failure  or  would  present  a  serious  risk  to  personnel  or  equipment,  and  (b)  UAS  operators  recently 
graduated  from  Advanced  Individual  Training  (AIT)  do  not  possess  these  skills.  Ratings  of  the 
skills  by  UAS  operators  and  helicopter  pilots  were  used  to  establish  an  ordering  of  the  skills  by 
training  criticality.  A  third  set  of  workshops  was  conducted  to  identify  perfonnance  indicators 
(i.e.,  benchmarks)  for  the  skills,  to  identify  events  and  processes  in  which  these  performance 
indicators  might  be  observed,  and  to  identify  locations  where  these  events  and  processes  might 
take  place.  All  workshops  were  conducted  at  Fort  Rucker,  AL. 


Defining  the  UAS  MUM-T  Task  and  Skill  Set 

The  initial  activities  in  this  project  focused  on  (a)  identifying  the  reconnaissance  and 
attack  missions  in  which  UAS  and  manned  aircraft  operate  as  a  team,  (b)  specifying  the  tasks 
required  perfonning  those  missions  that  involve  interactions  between  manned  aircraft  and  UAS, 
and  (c)  deriving  the  MUM-T  skills  that  are  needed  to  perform  these  tasks.  To  perform  these 
activities,  Army  doctrinal  material  and  regulations,  other  Army  documents  such  as  the  Army 
UAS  Roadmap  (U.S.  Army  UAS  Center  of  Excellence,  2010),  relevant  published  aviation 
training  research,  and  other  contemporary  sources  were  reviewed.  Based  on  this  infonnation,  a 
preliminary  list  of  missions,  tasks,  and  skills  was  produced.  A  workshop,  in  which  Army 
Aviation  SMEs  participated,  was  then  conducted  to  review  the  skills  list  and  to  make  any 
corrections,  additions,  and  clarifications. 

Identify  Missions 

Candidate  Anny  aviation  missions  were  derived  from  the  two  principal  manuals  for 
attack  helicopter  and  anned  reconnaissance  helicopter  operations  (i.e.,  Field  Manual  3-04.1 1 1, 
Aviation  Brigades,  Department  of  the  Army,  2007a;  Field  Manual  3-04.126,  Attack 
Reconnaissance  Helicopter  Operations;  Department  of  the  Army,  2007b).  Other  manuals  and 
regulations  also  contributed  to  the  wide  range  of  missions  considered.  The  process  used  to 
identify  specific  missions  involved  review  of  field  manuals,  Army  tactical  doctrine  in  general, 
and  literature  concerning  aviation  tactics  in  the  contemporary  environment.  The  list  of  missions 
for  consideration  was  based  on  an  expectation  that  UAS  operators  and  SR  and  attack  helicopter 
pilots  would  be  collaborating  as  intact  SR  teams  in  the  performance  of  all  mission  tasks. 

The  candidate  missions  were  categorized  into  groups  representing  reconnaissance,  attack, 
and  other  missions.  This  grouping  ensured  that  missions  that  used  similar  tasks  for  their 
execution  were  considered  together,  thus  simplifying  the  activity  of  identifying  tasks.  The 
information  from  these  manuals  and  regulations  was  analyzed  to  determine  which  of  these 
missions  were  currently  being  conducted  using  manned-unmanned  platform  teaming.  A  review 
of  contemporary  literature  on  helicopter  operations  led  to  modification  of  the  original  missions 
list  and  helped  anticipate  which  missions  might  be  supported  by  manned-unmanned  teams  in  the 
near-term  future.  A  list  of  the  missions  identified  in  this  assessment  is  presented  in  Appendix  A. 
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Identify  Tasks 

Following  Anny  doctrine,  MUM-T  tasks  were  defined  as  actions  needed  to  successfully 
execute  the  Aviation  tactical  missions.  To  identify  the  MUM-T  tasks,  a  list  of  UAS  tasks  was 
first  developed  by  analyzing  current  UAS  doctrine  and  scenarios  for  each  mission  to  detennine 
the  supporting  tasks  that  UAS  operators  could  be  expected  to  perform.  This  identification  was 
aided  by  review  of  contemporary  sources  describing  UAS  operations  during  recent  deployments 
and  on  projections  of  future  MUM-T  activities.  Contemporary  sources  examined  in  this  process 
included  aviation  or  other  military  news  publications  (e.g.,  Aviation  Week,  Defense  Daily), 
reports  by  Army  organizations  charged  with  the  development  of  aviation  capabilities  (e.g., 
Program  Executive  Office,  Aviation),  and  reports  published  by  large  military  contractors  and 
contractor  organizations  (e.g.,  Boeing,  National  Defense  Industrial  Association). 

MUM-T  tasks  were  selected  from  the  total  set  of  tasks  by  considering  scenarios  for  each 
Anny  aviation  mission  that  the  UAS  could  be  expected  to  support  and  by  analyzing  the  activities 
of  the  UAS.  The  focus  of  the  analysis  was  on  what  tasks  UAS  operators  were  expected  to  know 
and  perfonn.  The  analysis  was  aided  by  review  of  contemporary  literature  concerning  UAS 
operations  on  recent  deployments  and  by  ongoing  research  projects.  The  initial  MUM-T  task  list 
was  grouped  into  the  following  seven  categories  that  support  the  full  range  of  missions  defined 
by  this  review  of  Army  publications:  (a)  Conduct  aerial  observation,  (b)  Identify  threats,  (c) 
Share  targeting  infonnation,  (d)  Communicate  requirements,  (e)  Provide  battle  damage 
assessments,  (f)  Call  for  and  adjust  indirect  fires,  and  (g)  Conduct  cooperative  engagements. 

Identify  Skills 

Identification  of  critical  coordination  skills  that  Army  UAS  operators  need  for  MUM-T 
consisted  of  three  phases.  First,  published  materials  were  reviewed  to  create  a  preliminary  list  of 
UAS  operator  teaming  skills.  Second,  iterative  reviews  and  revisions  to  the  skill  list  were  made. 
Third,  the  skills  within  the  seven  task  categories  were  categorized  using  a  matrix  of  missions, 
tasks,  and  skills.  The  following  section  describes  the  activities  associated  with  each  phase  that 
resulted  in  the  identification  of  the  UAS  operator  coordination  skills. 

MUM-T  skills  are  the  behavioral  processes  UAS  operators  need  to  successfully  execute 
MUM-T  missions  and  tasks.  Identification  of  the  skills  involved  a  review  of  military  and 
research  documentation  on  UAS  capabilities  potentially  relevant  to  MUM-T  including  Anny 
field  manuals,  Programs  of  Instruction,  training  materials,  journal  articles,  and  other  government 
documents.  Table  2  lists  the  major  references  that  proved  most  useful  for  identifying  UAS 
coordination  skills  that  could  apply  to  MUM-T. 

Project  research  personnel,  who  were  SMEs  in  task  analysis,  reviewed  the  preliminary 
list  of  skills  and  detennined  that  not  all  of  the  skills  entailed  MUM-T.  Skills  that  were  not 
deemed  relevant  by  the  SMEs  were  excluded.  Further  modifications  of  the  skill  list  included 
revisions  to  wording,  removing  ill-defined  skills,  and  suggesting  additional  skills  for  inclusion. 
Revisions  and  updates  were  made  through  several  additional  iterations  to  produce  a  matrix  of 
skills  and  tasks  to  be  reviewed  by  SMEs.  The  matrix  linked  MUM-T  tasks  with  UAS  operator 
skills.  Skills  were  assigned  to  every  relevant  task  category.  Therefore  some  skills  were  listed 
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under  more  than  one  task  category.  Table  1  presents  the  major  technical  references  that  provided 
criteria  for  developing  these  skills  categorizations. 


Table  1 

Major  Reference  Sources  for  Manned-Unmanned  Teaming  Skills. 

Technical  References 

FM  3-04. 126.Attack  Reconnaissance  Helicopter  Operations  ( Department  of  the  Army,  2007b) 

FM  6-99.2.  U.S.  Army  report  and  message  formats.  (Department  of  the  Army,  2007c) 

FM  3-04.155.  Army  unmanned  aircraft  system  operations.  (Department  of  the  Army,  2009) 

TC  1-600.  Unmanned  aircraft  system  commander's  guide  and  aircrew  training 
manual  .(Department  of  the  Army,  2007d) 

TC  34-212.  Unmanned  aerial  vehicle  aircrew  training  manual .  (U.S.  Army  Intelligence 
Center,  1997) 

Notes:  FM  =  Field  Manual;  TC  =  Training  Circular. 


Task  and  Skill  Review  Workshop 

A  Task  and  Skill  Review  Workshop  was  conducted  to  obtain  feedback  from  UAS 
operators,  pilots,  and  training  and  doctrine  experts  on  the  draft  missions,  tasks,  and  skills  that 
were  produced  by  the  document  review.  A  focus  group  of  SMEs  reviewed,  revised,  and  edited 
the  draft  to  ensure  that  the  proposed  skills  were  actually  required  for  MUM-T.  The  focus  group 
was  also  asked  to  identify  any  additional  MUM-T  skills  not  included  in  the  list,  to  minimize 
overlap  among  skill  descriptions,  and  to  discuss  prerequisite  relationships  and  sequential 
dependencies  among  skills. 

Method 

Participants.  Of  the  eight  participants  in  the  Task  and  Skill  Review  Workshop,  four 
were  from  the  U.S.  Anny  Aviation  Center  of  Excellence  Directorate  of  Training  and  Doctrine 
(DOTD),  two  were  from  the  Training  and  Doctrine  Command  Capabilities  Manager  (TCM)  for 
UAS,  and  the  remaining  two  were  from  the  U.S.  Army  Aviation  Center  of  Excellence 
Noncommissioned  Officer  Academy.  The  participants  included  four  noncommissioned  officers, 
one  chief  warrant  officer,  one  commissioned  officer,  and  two  U.S.  Army  civilians.  All  six 
military  personnel  had  completed  a  tour  in  a  combat  zone  in  the  previous  3  years.  Five  had  been 
in  Iraq,  and  one  of  these  had  also  served  in  Afghanistan  (one  did  not  indicate  an  area  of 
operations).  Four  of  the  military  personnel  were  UAS  operators  qualified  in  medium  UAS  (e.g., 
RQ-7B,  MQ-5B).  The  remaining  two  were  OH-58D  anned  scout  helicopter  pilots.  Five 
participants  indicated  that  they  had  experience  with  reconnaissance/security  missions.  One  pilot 
indicated  additional  experience  with  attack  missions. 

Materials.  An  interview  protocol  was  drafted  that  included  a  set  of  twenty  discussion 
questions  about  the  MUM-T  skills  list  constructed  in  the  previous  document  review.  The 
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discussion  questions  are  listed  in  Appendix  B.  The  Participant  Background  Form  asked  seven 
questions  to  gauge  the  expertise  and  experiences  of  the  workshop  participants  in  the  MUM-T 
environment,  and  is  shown  in  Appendix  C.  Several  MUM-T  scenarios  were  derived  from  Army 
field  manuals  for  reconnaissance  and  attack  missions.  These  scenarios  were  provided  to 
participants  during  the  workshop  to  facilitate  discussion  about  the  required  skills  for  UAS 
operators  in  the  MUM-T  environment. 

Procedure.  The  workshop  began  with  an  overview  of  the  project  and  the  specific  goals 
for  the  workshop.  Informed  consent  and  Participant  Background  Forms  were  also  distributed  to 
participants  in  this  and  all  subsequent  workshops,  in  compliance  with  ARI  ethical  protocol 
concerning  use  of  human  participants.  The  topic  of  the  discussion  then  turned  to  the  list  of 
missions,  tasks,  and  skills  for  UAS  operators  identified  prior  to  the  workshop.  In  each  case,  the 
SMEs  were  asked  whether  items  (i.e.,  missions,  task,  or  skills)  should  be  added  or  removed  from 
the  list  and  whether  the  items  were  valid  in  terms  of  actual  practice  in  the  field.  The  SMEs  were 
also  asked  whether  there  were  likely  future  missions,  tasks,  or  skills  that  should  be  considered  in 
this  research.  SMEs  were  asked  whether  some  skills  were  prerequisites  for  others,  and  about  the 
sequential  performance  dependencies  between  identified  skills.  SMEs  made  several  additions 
and  deletions  to  the  list  of  skills.  In  addition,  they  reviewed  and  edited  terminology  used  to 
describe  the  items. 

Results 

SMEs  did  not  recommend  any  changes  to  the  list  of  missions,  but  they  did  recommend 
several  changes  to  the  identified  skills  and  some  changes  to  the  tasks.  These  varied  from 
relatively  minor  rewording  of  a  task  or  skill  to  major  changes  in  which  skills  were  added, 
deleted,  or  combined.  The  following  list  summarizes  the  types  of  changes  that  were  made  to 
skill  and  task  statements  presented  in  Table  2. 

•  Added  skills.  SMEs  added  skills  that  they  believed  were  missing  from  the  original  list. 
For  example,  they  added  the  skill,  “deconflict  munition  trajectories  from  airframes,”  to 
the  task,  “call  for  and  adjust  indirect  fires.” 

•  Changes  in  scope  of  tasks  or  skills.  SMEs  enlarged  or  focused  the  scope  of  some  tasks  or 
skills.  For  example,  the  task,  ’’identify  threats  to  aerial  maneuver”  was  changed  to 
“identify  threats  to  maneuver,”  because  the  SMEs  judged  that  a  UAS  operator  would  be 
required  to  identify  threats  to  ground  maneuver  as  well  as  aerial  maneuver. 

•  Combining  redundant  or  similar  skills.  SMEs  developed  general  skills  to  encompass 
several  closely  related  skills  in  the  original  list.  For  example,  they  identified  a  general 
skill,  “utilize  standardized  radio  communication  and  signal  operating  procedures,”  to 
combine  what  had  been  represented  in  three  communication  skills  in  the  draft  list. 

•  Adding  or  removing  associations  between  skills  and  tasks.  For  example,  the  skill,  “call 
for  indirect  fires”  was  removed  from  cooperative  engagement  task,  because  it  was  judged 
to  be  inappropriate  for  that  task. 
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•  Clarifying  terminology.  SMEs  refined  or  clarified  terms  used  in  task  and  skills,  such  as 
the  distinction  between  high  payoff  targets  and  high  value  targets. 

•  Editorial  changes.  SMEs  made  several  changes  to  standardize  descriptions  of  skills  and 
to  incorporate  appropriate  military  tenns. 


Table  2 

Task  Categories  and  Associated  Manned-Unmanned  Teaming  Skills 


Task  Category  MUM-T  Skill 

Communicate  Requirements  Utilize  standardized  radio  communication  and  signal  operating 

procedures* 

Utilize  standardized  report  formats 

Transmit  imagery,  sensor  data,  tactical  situational  maps, 
overlays,  and  reports  (e.g.,  spot  reports)* 

Develop  and  send  common  operating  picture  information  to  air- 
ground  team 

Utilize  joint,  Army,  and  civilian  personnel  recovery  terminology 
Conduct  Cooperative  Engagements  Find/track  targets  (e.g.,  HPTs,  HVTs)* 

Provide  the  direction  of  the  target  in  degrees  and  range  from  the 
battle  position* 

Transmit  information  about  the  method  of  attack  (i.e.,  scheme 
of  maneuver,  fire  distribution,  and  maneuver  for  the  attack)* 

Transmit  imagery,  sensor  data,  tactical  situational  maps, 
overlays,  and  reports  (e.g.,  spot  reports)* 

Select  the  best  weapon  systems  to  engage  the  target  (e.g., 
Icthal/nonlcthal,  munitions  effect,  collateral  damage  assessment) 

Transmit  information  about  the  location  of  threat  forces,  terrain, 
and  obstacles  that  influence  operations* 

Provide  confirmation  of  the  target  prior  to  engagement* 

Identify  Threats  Find/track  targets  (e.g.,  FIPTs,  FIVTs)* 

Transmit  information  about  the  location  of  threat  forces,  terrain, 
and  obstacles  that  influence  operations* 

Transmit  imagery,  sensor  data,  tactical  situational  maps, 
overlays,  and  reports  (e.g.,  spot  reports)* 

Provide  accurate  description  of  target  to  support  target  selection 
Provide  confirmation  of  the  target  prior  to  engagement* 
Prioritize  the  engagement  of  targets 

Notes'.  Asterisk  (*)  indicates  skills  that  appear  in  multiple  task  categories;  HPT  =  high-payoff  target; 
HVT  =  high-value  target. 
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Table  2  (continued) 

Task  Categories  and  Associated  Manned-Unmanned  Teaming  Skills 


Task  Category 

MUM-T  Skill 

Conduct  Aerial  Observation 

Gain  and  maintain  enemy  contact 

Provide  early  warnings,  ambush  detection,  overwatch,  threat 
identification 

Transmit  information  about  the  location  of  threat  forces, 
terrain,  and  obstacles  that  influence  operations* 

Transmit  imagery,  sensor  data,  tactical  situational  maps, 
overlays,  and  reports  (e.g.,  spot  reports)* 

Share  Targeting  Information 

Provide  the  target  location  (i.e.,  direction  of  target  in  degrees 
and  range  from  battle  position) 

Transmit  information  about  the  location  of  threat  forces, 
terrain,  and  obstacles  that  influence  operations* 

Provide  target  description  information* 

Transmit  infomiation  about  the  method  attack  (i.e.,  scheme  of 
maneuver,  fire  distribution,  and  maneuver  for  the  attack)* 

Utilize  standardized  execution  commands  to  initiate  attack 

Conduct  target  handover 

Call  for  and  Adjust  Indirect  Fires 

Provide  the  direction  of  the  target  in  degrees  and  range  from 
the  battle  position* 

Provide  target  description  information* 

Deconflict  munition  trajectories  from  airframes 

Conduct  call  for  indirect  fires 

Provide  Battle  Damage  Assessment 

Utilize  standardized  radio  communication  and  signal 
operating  procedures* 

Transmit  imagery,  sensor  data,  tactical  situational  maps, 
overlays,  and  reports  (e.g.,  spot  reports)* 

Perform  battle  damage  assessment 

Notes :  Asterisk  (*)  indicates  skills  that  appear  in  multiple  task  categories;  HPT  =  high-payoff  target; 
HVT  =  high-value  target. 

For  the  seven  task  categories  presented  in  Table  2,  a  total  of  25  coordination  skills  were 
identified  as  required  by  UAS  operators  for  MUM-T.  Table  2  lists  the  skills  for  each  task  in  the 
sequence  they  would  typically  be  performed  based  on  the  judgments  expressed  by  the  SMEs. 

For  example,  a  target  must  be  located  before  it  can  be  engaged  with  direct  or  indirect  fire.  Not 
all  skills  would  be  required  to  be  executed  within  a  typical  SR  mission.  These  skills  are  shown 
along  with  the  corresponding  MUM-T  task(s)  with  which  each  skill  is  associated.  Appendix  D 
provides  a  list  of  references  within  published  U.S.  Army  directives  and  regulations  defining  each 
of  the  identified  skills. 


8 


Training-Critical  MUM-T  Skills  Workshop 


MUM-T  skills  will  be  critical  to  train  when  (a)  UAS  operators  who  have  completed  AIT 
do  not  possess  the  skills,  (b)  the  skills  are  required  for  successful  perfonnance  of  multiple 
missions,  and  (c)  insufficient  skill  levels  can  produce  a  serious  increase  in  risk  to  personnel  and 
equipment.  Training  criticality  variables  were  assessed  as  a  function  of  the  current  training 
system  and  the  expected  perfonnance  of  UAS  operators  at  the  completion  of  AIT.  Thus, 
criticality  reflects  the  need  for  additional  training  beyond  what  is  currently  provided.  If  a  skill  is 
trained  sufficiently  well,  so  that  UAS  operators  and  pilots  can  effectively  perfonn  their 
reconnaissance  and  attack  missions  and  avoid  risk  to  personnel  and  equipment,  then  training 
criticality  is  low  for  that  skill,  regardless  of  the  importance  of  the  skill  to  the  mission  or  the  risks 
of  poor  performance.  For  skills  that  are  not  trained  in  AIT  or  are  covered  more  superficially, 
importance  to  the  mission  and  risks  of  poor  performance  have  a  greater  impact  on  the  overall 
training  criticality  score.  The  time  required  to  acquire  a  skill  also  contributes  to  skill  criticality 
when  a  limited  training  budget  must  be  allocated  to  train  a  subset  of  the  total  complement  of 
required  skills.  In  that  case,  skills  that  can  be  quickly  trained  represent  a  more  efficient  use  of  the 
training  budget  with  all  other  criteria  being  equal. 

In  this  phase  of  the  research,  a  group  of  experienced  UAS  operators  and  helicopter  pilots 
individually  rated  each  of  the  25  MUM-T  skills  identified  in  the  first  workshop.  The  skills  were 
rated  on  four  dimensions  related  to  training  criticality  (i.e.,  performance,  importance  as  to 
mission  [attack,  reconnaissance],  and  to  personnel  and  equipment.)  as  well  as  on  the  level  of 
resources  required  for  training  each  skill.  The  original  intention  had  been  to  obtain  ratings  in 
group  workshops,  which  would  allow  additional  information  to  be  obtained  in  support  of  the 
rationale  for  the  ratings.  However,  scheduling  groups  of  participants  was  infeasible,  so  the 
ratings  were  individually  obtained.  Ratings  were  obtained  using  spreadsheets  with  a  guided  user 
interface  presented  on  laptop  computers. 

Method 

Participants.  A  total  of  23  raters  provided  judgments,  including  18  UAS  operators  and 
five  helicopter  pilots.  All  but  two  of  the  23  raters  had  completed  at  least  one  tour  in  a  combat 
zone  within  the  last  3  years,  and  those  tours  were  mostly  in  Iraq  or  Afghanistan.  All  raters  were 
experienced  with  reconnaissance/security  missions.  Seventeen  were  experienced  with 
communication/relay  missions  and  1 1  were  experienced  with  attack  missions.  All  18  of  the  UAS 
operators  were  qualified  to  operate  a  medium  UAS,  such  as  the  Shadow,  while  five  were  also 
qualified  to  operate  a  heavy  UAS,  such  as  the  MQ-1C.  The  UAS  operators  were  primarily  senior 
noncommissioned  officers;  pilots  were  warrant  officers.  The  5  raters  who  were  pilots  were 
qualified  to  operate  the  OH-58D  anned  scout  helicopter. 

Materials.  The  assessment  of  training  criticality  was  based  primarily  on  the  Task  and 
Training  Requirements  Analysis  Methodology  (Swezey,  Owens,  Bergondy,  &  Salas,  1998)  with 
some  modifications  as  described  in  the  following  discussion.  In  addition,  a  measure  of  the 
resources  required  to  train  each  skill  was  incorporated  into  the  assessment.  Training  criticality 
was  assessed  using  the  following  four  dimensions:  Performance  Level,  Importance  to  attack  and 
reconnaissance  missions,  and  Consequences  of  lack  of  skill  {to  personnel  and  equipment).  A 
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fifth  dimension,  training  resources,  was  an  ancillary  measure  that  did  not  comprise  the  main  index  of 
training  criticality.  These  dimensions  are  their  criteria  are  described  below: 

Performance  level.  Performance  level  was  based  on  a  rating  scale  developed  by 
Campbell,  et  al.  (1990).  Raters  were  asked  to  reflect  on  the  knowledge  and  skills  typical  of  UAS 
operators  who  had  just  completed  AIT,  based  on  their  own  operational  experiences  and  training. 
They  were  then  asked  to  allocate  a  hypothetical,  unspecified  group  of  10  typical  AIT  graduates 
into  five  proficiency  categories  for  each  of  the  25  skills  identified  previously. 

1 .  The  number  of  Soldiers  that  Virtually  Never  perform  the  skill  effectively; 

2.  The  number  of  Soldiers  that  perfonn  the  skill  effectively  Less  Than  Half  the 
Time; 

3.  The  number  of  Soldiers  that  perfonn  the  skill  effectively  About  Half  the  Time; 

4.  The  number  of  Soldiers  that  perfonn  the  skill  effectively  Most  of  the  Time; 

5.  The  number  of  Soldiers  that  perfonn  the  skill  effectively  All  of  the  Time. 

The  spreadsheet  on  which  raters  responded  automatically  checked  to  ensure  that  the  total  number 
of  entries  was  10  for  each  skill. 

The  mean  category  to  which  the  Soldiers  were  assigned  represented  the  average  skill 
level.  Skilled  perfonnance  could  occur  either  because  Soldiers  were  well  trained  on  this  skill  in 
AIT  or  because  the  skill  was  easy  or  already  known  by  entering  Soldiers.  The  standard  deviation 
of  the  response  allocation  represents  the  performance  variability  of  the  skill.  Perfonnance 
variability  has  a  maximum  value  of  approximately  2.0  when  five  Soldiers  are  assigned  to  the 
lowest  category  (Virtually  Never),  while  the  remaining  5  Soldiers  are  assigned  to  the  highest 
category  (All  of  the  Time).  When  all  Soldiers  are  assigned  to  the  same  category,  the 
performance  variability  reaches  its  minimum  value  of  zero.  A  uniform  distribution  of  Soldiers 
among  the  five  categories  produces  a  perfonnance  variability  of  1.41. 

Importance  to  success  of  attack  missions.  Raters  were  asked  to  indicate  the  extent  to 
which  the  skills  were  important  to  the  success  of  attack  missions.  Responses  were  made  on  a  5- 
point  scale  with  anchors  for  the  extreme  and  middle  points.  Thus,  the  following  response  scale 
was  used: 

1 .  Low.  Lack  of  skill  unlikely  to  have  negative  consequence  on  perfonnance  of 
attack  mission.  Skill  is  not  critical  to  attack  mission, 

2.  Low-Moderate, 

3.  Moderate.  Lack  of  skill  would  have  moderate  consequences  and  might 
jeopardize  attack  mission  success.  Skill  is  somewhat  important  to  attack  mission, 

4.  Moderate-High, 

5.  High.  Lack  of  skill  would  most  likely  have  serious  consequences  and  lead  to 
attack  mission  failure.  Skill  is  critical  to  attack  mission. 

Importance  to  success  of  reconnaissance  missions.  Raters  were  asked  to  indicate  the  extent  to 
which  the  skills  were  important  to  the  success  of  reconnaissance  missions.  The  fonnat  of  the 
question,  response  options  and  5-point  scale  were  the  same  as  for  attack  missions. 
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Consequences  to  personnel  and  equipment.  Raters  were  asked  to  rate  the  extent  to 
which  the  skills  were  important  to  reduce  risk  to  personnel  and  equipment.  Responses  were 
made  on  a  5-point  scale  with  anchors  for  the  extreme  and  middle  points.  The  anchors  varied 
both  the  likelihood  and  severity  of  consequences  of  lack  of  skill.  The  following  response  scale 
was  used: 

1.  Low.  Lack  of  skill  unlikely  to  increase  risk  to  personnel  and  equipment, 

2.  Low-Moderate, 

3.  Moderate.  Lack  of  skill  would  produce  a  moderate  increase  in  risk  to  personnel 
and  equipment, 

4.  Moderate-High, 

5.  High.  Lack  of  skill  would  most  likely  produce  a  serious  increase  in  risk  to 
personnel  and  equipment. 

Training  resources.  Although  not  one  of  the  four  dimensions  directly  comprising  the 
training  criticality  measure,  raters  were  asked  to  indicate  the  level  of  resources  required  to  train  a 
UAS  operator  to  mission  proficiency  on  each  of  the  25  MUM-T  skills.  These  ratings  were 
included  because  resources  are  often  limited  and  must  be  allocated  to  the  most  critical  tasks. 
Having  an  estimate  of  the  resources  required  would  allow  calculation  of  the  ratio  of  training 
criticality  to  resources.  The  resource  ratio  could  provide  a  useful  guide  for  specifying  how 
resources  should  be  allocated  to  address  the  most  critical  tasks  within  a  resource  budget.  The 
raters  were  asked  to  rate  each  of  the  skills  according  to  the  amount  of  training  resources  (time, 
personnel,  and/or  equipment)  required  to  train  it.  Responses  were  made  on  a  5 -point  scale  with 
anchors  for  the  extreme  and  middle  points,  using  the  following  response  scale: 

1 .  Low.  Skill  requires  minimal  training  time  and  little  or  no  use  of  personnel  or 
equipment  to  achieve  an  effective  perfonnance  level, 

2.  Low-Moderate, 

3.  Moderate.  Skill  requires  moderate  training  time  and  some  use  of  personnel  and 
equipment  to  achieve  an  effective  perfonnance  level, 

4.  Moderate-High, 

5.  High.  Skill  requires  extensive  training  time,  personnel,  and  equipment  to  achieve 
an  effective  perfonnance  level. 

Procedure.  The  workshop  began  with  the  participants  completing  Infonned  Consent 
Forms  and  the  Participant  Background  Forms  (Appendix  C).  When  these  forms  were  completed, 
the  participants  then  rated  each  of  the  25  MUM-T  skills.  Ratings  were  conducted  using  a 
Microsoft  Excel  spreadsheet,  which  provided  feedback  to  the  rater  to  ensure  that  they  properly 
rated  all  skills  for  each  of  the  four  dimensions.  Raters  rated  all  skills  for  one  dimension  and  then 
proceeded  to  the  next  dimension  until  all  skills  dimensions  were  completed. 


Results 

Summary  of  ratings.  Average  perfonnance  level  estimates  (expressed  as  percentages 
rounded  to  the  nearest  percentage  point)  are  shown  in  Table  3.  For  each  rater,  the  mean 
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performance  level  rating  and  perfonnance  variability  for  each  MUM-T  skill  was  calculated.  The 
mean  of  these  values  over  raters  indicated  that  skills  vary  greatly  in  rated  performance  with  mean 
ratings  ranging  from  a  minimum  of  1.39  (indicating  that  most  newly  trained  operators  virtually 
never  perfonned  the  skill  correctly)  to  a  maximum  of  4.26  (indicating  that  most  operators 
performed  the  skill  correctly  all  or  most  of  the  time).  A  total  of  8  of  the  25  skills  had  mean 
performance  ratings  of  less  than  2.0,  which  indicated  that  the  typical  newly  trained  UAS 
operators  were  rated  to  perfonn  these  skills  incorrectly  most  of  the  time.  Average  performance 
variability  was  low  with  a  maximum  value  of  0.36  on  a  scale  ranging  from  0.0  to  2.0.  This 
suggested  that  low  performance  variability  implied  that  raters  tended  to  place  most  of  the  10 
hypothetical  Soldiers  in  the  same  or  similar  categories  rather  than  spreading  their  ratings  over  the 
scale.  The  low  performance  variability  may  reflect  the  fact  that  the  hypothetical  Soldiers  were 
expected  to  have  received  the  same  training  at  AIT  and  that  no  differences  in  on-the-job 
experience  would  distinguish  them. 

The  mean  importance  ratings  are  shown  in  Table  4  and  indicated  that  all  the  skills  were 
judged  to  be  at  least  moderately  important  to  attack  missions,  to  reconnaissance  missions  and  to 
present  moderate  to  serious  risks  to  personnel  and  equipment  if  incorrectly  performed.  Mean 
importance  of  skills  to  attack  missions  varied  from  2.96  to  4.74  (5-point  scale).  Similarly,  mean 
importance  to  reconnaissance  missions  varied  from  2.91  to  4.74.  For  each  of  these  variables,  10 
of  the  25  skills  had  mean  ratings  greater  than  4.0.  Ratings  of  the  severity  of  consequences  of 
lack  of  skill  for  personnel  and  equipment  ranged  from  3. 17  to  4.39  with  9  of  25  skills  receiving 
mean  ratings  of  greater  than  4.0.  Intercorrelations  between  mean  ratings  of  all  three  importance 
dimensions  were  statistically  significant  a  <  .01  (r  attack/  reconnaissance  (23)  =  .86;  r  attack/ personnel  & 

equipment  (23)  .96,  1  (23)  reconnaissance/personnel  &  equipment  .88).  These  high  Correlations  Suggested 

that  the  three  dimensions  are  equivalent,  likely  redundant,  and  reinforced  the  rationale  for 
weighting  them  equally.  Finally,  it  is  noteworthy  that  mean  estimated  perfonnance  levels 
significantly  correlated  with  importance  dimensions  for  attack,  reconnaissance,  and  personnel 
and  equipment  at  the  a  <  .01  level  (r  (23)  =[  .57,  .72,  .56],  respectively),  which  indicated  higher 
perceived  perfonnance  for  those  skills  adjudged  most  important.  The  latter  lower  conelations 
seem  reasonable,  as  estimated  performance  showed  more  variability  than  ratings  of  importance. 

The  mean  rating  for  training  resources  was  3.38  (SD=. 30).  This  value  was  slightly 
greater  than  the  moderate  option  on  the  5-point  scale.  Ratings  ranged  between  2.78  and  3.91. 
Despite  this  relatively  restricted  range,  it  was  possible  that  the  ratings  for  this  variable  could  be 
used  to  identify  critical  skills  that  could  be  trained  using  a  relatively  modest  amount  of  resources. 
For  example,  those  skills  that  call  for  the  use  of  proper  communication  procedures  and  fonnats, 
that  do  not  require  expensive  training  device  support,  could  be  trained  at  home  station  on  PC- 
based  media.  However,  training  resources  was  regarded  simply  as  an  ancillary  measure  of 
potential  utility  to  the  training  developer  and  did  not  contribute  to  the  determination  of  criticality. 
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Table  3 

Estimated  Performance  Levels  for  25  Manned-Unmanned  Teaming  Skills. 


Skill 

Estimated  Percent  per  Each 

Performance 

Level  of  Performance 

Rating 

1 

2 

3 

4 

5 

Mean 

SD 

Gain  and  maintain  enemy  contact 

24 

15 

31 

20 

10 

2.91 

0.36 

Provide  early  warnings,  ambush  detection, 
overwatch,  threat  identification 

4 

34 

15 

38 

9 

3.22 

0.36 

Transmit  information  about  location  of  threat 

13 

34 

7 

18 

28 

3.13 

Transmit  imagery,  sensor  data,  etc 

21 

5 

12 

22 

40 

3.57 

Find  and  track  targets 

1 

2 

31 

33 

33 

3.96 

Provide  accurate  description  of  target 

3 

2 

37 

35 

23 

3.74 

0.31 

Provide  confirmation  of  target 

7 

4 

50 

30 

9 

3.48 

Prioritize  engagement  of  targets 

63 

20 

7 

5 

5 

1.78 

Provide  target  location 

29 

4 

13 

20 

34 

3.39 

Provide  target  description 

4 

4 

2 

59 

31 

4.26 

Transmit  information  on  method  of  attack 

69 

6 

19 

6 

0 

1.65 

Utilize  standard  execution  commands  to 
initiate  attack 

67 

11 

11 

7 

4 

1.74 

0.11 

Conduct  target  handover 

7 

45 

21 

10 

17 

3.00 

0.35 

Utilize  standardized  radio  communication 

4 

27 

10 

25 

34 

3.70 

0.31 

Utilize  standardized  report  formats 

4 

12 

10 

39 

35 

3.96 

0.25 

Develop/send  common  operating  picture 
information 

67 

11 

10 

8 

4 

1.78 

0.12 

Utilize  joint,  Army,  &  civilian  personnel 
recovery  terminology 

70 

12 

5 

5 

8 

1.65 

0.17 

Perform  battle  damage  assessment 

27 

6 

6 

23 

38 

3.52 

Provide  direction  of  target 

30 

4 

8 

25 

33 

3.39 

Deconflict  munition  trajectories  from  airframe 

83 

6 

8 

3 

0 

1.39 

Conduct  call  for  indirect  fires 

42 

9 

10 

24 

15 

2.52 

Select  best  weapon  system 

87 

3 

1 

3 

6 

1.39 

Transmit  information  about  location  of  unit  of 
action 

1 

16 

11 

37 

35 

4.04 

0.23 

Conduct  call  for  direct  fires 

66 

5 

9 

15 

5 

1.91 

0.25 

Switch  roles  of  laser  designator 

70 

5 

4 

8 

13 

2.00 

0.18 

Note.  Performance  Levels:  1=  Virtually  Never;  2 
4=Most  of  the  Time;  5=  All  of  the  Time. 


Less  than  Half  the  Time;  3 


About  Half  the  Time; 
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Training  Criticality  Index.  A  training  criticality  index  (TCI)  was  created  to  combine 
the  perceived  performance  level  ratings  (Table  3)  with  the  dimensions  of  importance  to  missions 
(i.e.,  attack  and  reconnaissance),  and  importance  to  personnel  and  equipment  (Table  4).  First, 
the  index  included  an  additive  combination  of  the  ratings  of  the  three  dimensions  of  importance. 
Given  the  intercorrelations  of  these  three  dimensions,  the  three  importance  ratings  were  equally 
weighted  in  detennining  the  index.  Second,  the  index  included  a  multiplicative  combination  of 
perfonnance  level  with  the  aggregated  importance  dimensions.  If  all  Soldiers  judged  to  be  able 
to  adequately  perform  a  skill  (that  is,  the  skill  is  rated  to  be  in  category  5  on  the  perfonnance 
scale),  then  the  training  criticality  is  low  because  there  is  not  a  need  for  additional  training. 
However,  as  average  perfonnance  decreases,  the  importance  of  consequences  of  lack  of  skill 
increases  correspondingly.  A  training  criticality  index  that  has  the  previous  properties  is  given 
by  the  following  equation: 


rc;  =  cs-Pi)x(cx  +  c,  +  cM), 

where  TCI  is  the  training  criticality  index,  PL  is  the  performance  level  rating  (on  a  scale  from  1 
to  5),  Ca,  Cr,  and  CPE  are  the  consequences  of  lack  of  skill  (i.e.,  importance)  to  the  attack 
mission,  the  reconnaissance  mission,  and  personnel  and  equipment,  respectively  (each  also  on  a 
scale  from  1  to  5).  That  is,  the  greater  the  potential  consequences,  the  greater  the  importance  of 
the  skill  is  to  the  mission.  The  TCI  function  has  a  minimum  value  of  zero  when  either  of  the  two 
factors  is  zero,  and  has  a  maximum  value  of  60,  when  the  performance  level  is  at  its  minimum 
value  and  the  consequence  variables  are  all  at  their  maximum  levels.  For  a  worked  example,  let 
us  compute  the  TCI  for  the  skill:  deconflict  munition  trajectories  from  airframe.  The  mean 
estimated  perfonnance  level  was  1.39;  mean  importance  ratings  were  CA  =  3.87;  CR  =  3.17,  and 
CPE  =  3.74).  TCI  =  (5-1.39)  x  (3.87+3.17+3.74)  =  3.61  x  10.78  =  38.92.  This  skill  was  highest 
in  training  criticality  of  all  25  of  skills  identified. 

The  last  column  in  Table  4  shows  the  skills  ordered  by  their  value  on  the  TCI.  Because 
the  mean  perfonnance  level  varies  more  than  the  three  importance  measures,  it  is  not  surprising 
that  the  most  training-critical  skills  are  the  ones  in  which  perceived  or  expected  performance  or 
new  AIT  graduates  was  rated  lowest.  Examination  of  the  most  training-critical  skills  shows  that 
nearly  all  of  them  focus  on  tactical  activities  required  for  reconnaissance  and  attack  missions. 
Skills  at  the  bottom  of  the  list  have  a  low  training-criticality  index  despite  their  high  importance 
because  Soldiers  graduating  from  AIT  are  seen  as  already  knowing  these  skills  well.  Recall  that 
the  lowest  ranking  skill,  Provide  target  description  information,  was  rated  as  being  performed 
correctly  most  or  all  of  the  time  by  90%  of  newly  trained  UAS  operators  (Table  4).  Because 
rated  perfonnance  is  already  so  high  on  this  important  skill,  it  is  not  critical  to  provide  training 
on  it  beyond  AIT.  Review  of  the  less  training-critical  skills  (those  with  TCI  scores  <  20.0) 
shows  that  most  of  them  are  consistent  with  the  traditional  MI  mission  for  UAS.  Again,  these 
are  very  important  skills  in  terms  of  impact  on  mission  success,  but  they  are  not  training-critical 
as  they  are  adequately  trained  during  AIT. 
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Table  4 

Importance  Ratings  for  25  Manned-Unmanned  Teaming  Skills  Rank-Ordered  of  by  Training 
Criticality  Index  Score. 


Rank  Skill 

Importance 

TCI 

Attack 

Recon 

Personnel 

and 

Equipment 

1  Deconflict  munition  trajectories  from  airframe 

3.87 

3.17 

3.74 

38.92 

Utilize  standard  execution  commands  to  initiate 
^  attack 

3.65 

3.39 

3.61 

34.74 

3  Transmit  information  on  method  of  attack 

3.39 

3.26 

3.61 

34.35 

4  Switch  roles  of  laser  designator 

4.04 

3.52 

3.70 

33.78 

5  Conduct  call  for  direct  fires 

3.57 

3.61 

3.70 

33.55 

6  Select  best  weapon  system 

3.13 

2.91 

3.17 

33.26 

Develop/send  common  operating  picture 
'  information 

3.35 

3.35 

3.30 

32.17 

Utilize  joint.  Army,  &  civilian  personnel  recovery 
^  terminology 

2.96 

3.17 

3.22 

31.29 

9  Prioritize  engagement  of  targets 

3.00 

3.30 

3.30 

30.91 

10  Gain  and  maintain  enemy  contact 

4.74 

4.52 

4.39 

28.49 

1 1  Conduct  call  for  indirect  fires 

3.91 

3.57 

3.83 

28.02 

12  Transmit  information  about  location  of  threat 

4.52 

4.74 

4.30 

25.36 

13  Conduct  target  handover 

4.00 

4.48 

3.87 

24.70 

Provide  early  warnings,  ambush  detection, 

^  overwatch,  threat  identification 

4.52 

4.61 

4.35 

24.03 

15  Provide  target  location 

4.30 

4.26 

4.30 

20.70 

16  Provide  confirmation  target 

4.43 

4.39 

4.17 

19.78 

17  Provide  direction  of  target 

4.26 

3.96 

3.96 

19.58 

lg  Transmit  imagery,  sensor  data,  etc 

3.96 

4.43 

4.04 

17.84 

19  Provide  accurate  description  of  target 

4.43 

4.43 

4.17 

16.45 

20  Utilize  standardized  radio  communication 

3.91 

3.87 

3.83 

15.14 

21  Perform  battle  damage  assessment 

3.26 

3.30 

3.22 

14.46 

22  Find  and  track  targets 

4.48 

4.48 

4.39 

13.93 

Transmit  information  about  location  of  unit  of 
action 

3.87 

3.96 

3.74 

11.06 

24  Utilize  standardized  report  formats 

3.39 

3.74 

3.30 

10.89 

25  Provide  target  description  information 

4.26 

4.22 

4.30 

9.45 

Grand  Mean 

SD 

3.89 

3.87 

3.82 

24.11 

0.52 

0.55 

0.41 

8.87 

Note:  TCI  =  Training  Criticality  Index 
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Benchmark  Indicators  Workshop 


A  benchmark  is  a  standard  by  which  something  can  be  measured  or  judged.  It  is  not 
itself  an  assessment  method  but  rather  a  descriptor  or  locator  of  a  particular  (usually  desired) 
level  of  an  attribute  or  behavior.  In  the  case  of  a  training  system,  benchmarks  can  be  used  to 
provide  instructional  feedback  and  diagnostics  during  training,  inform  the  evaluation  of  a  course 
or  program,  or  assess  instructor  performance.  Benchmarks  can  also  be  used  as  indices  of  levels 
of  proficiency  including  minimal  competence.  A  benchmark  is  usually  a  location  on  a 
continuum  derived  from  an  observational  measure  of  performance.  A  benchmark  assumes  an 
observable  continuum  to  be  measured,  a  method  for  measuring  it,  and  the  establishment  of  a 
point  on  that  continuum  that  corresponds  to  the  level  of  the  attribute  that  is  of  interest.  For  the 
current  research  effort,  the  prime  emphasis  for  the  purpose  of  benchmarks  is  in  determining 
minimum  competence,  and  the  secondary  emphasis  is  on  instructional  feedback.  Therefore, 
benchmarks  could  be  considered  as  both  indicators  of  competent  perfonnance  and  indicators  of 
non-competent  performance. 

The  approach  to  developing  benchmarks  for  previously  identified  MUM-T  skills  set  was 
to  elicit  relevant  skill  descriptions  from  experienced  individuals  representing  both  the  manned 
aviation  and  unmanned  aviation  communities.  The  descriptions  included  behavioral  indicators 
of  competency  (or  non-competency),  conditions  under  which  those  indicators  would  be 
observable,  and  environments  in  which  the  observable  behaviors  would  be  likely  to  occur.  This 
approach  was  implemented  through  two  group-discussion  workshops  using  brief  written 
response  fonns  and  consensus  oriented  discussion.  The  reader  should  note  that  workshop 
participants  were  not  expected  to  be  experts  in  instructional  design  or  behavioral  sciences. 
Therefore,  it  was  not  expected  for  participants  to  be  able  to  identify  or  formulate  performance 
benchmarks  within  a  training  program.  However,  it  was  expected  that  participants  would  have 
technical  expertise  in  UAS  operations  (possibly  to  include  MUM-T)  and  could  describe 
behavioral  indices  for  competency  in  MUM-T,  especially  if  participants  had  direct  past 
experience  in  MUM-T. 

Method 

Participants.  Participants  were  recruited  from  staff  positions  and  from  senior  non¬ 
commissioned  officer  training  courses  at  the  U.S.  Army  Aviation  Center  of  Excellence.  The 
recruiting  effort  called  for  personnel  who  had  direct  experience  in  missions  relevant  to  MUM-T 
and  preferably  direct  experience  in  MUM-T.  Due  to  scheduling  limitations  and  in  order  to  keep 
group  sizes  to  a  manageable  level,  separate  workshops  were  conducted  with  one  workshop 
consisting  of  manned  aircraft  pilots  and  the  second  workshop  consisting  of  UAS  operators. 

There  were  12  participants  in  the  two  workshops. 

In  the  first  workshop,  five  chief  warrant  officers  took  part.  All  were  OH-58D  qualified, 
and  all  had  experience  in  combat  zones  in  Iraq  as  well  as  other  areas  of  operation.  All  had 
experience  in  reconnaissance/security  missions.  Two  had  experience  in  attack  missions,  and  one 
had  experience  in  communications  missions.  Most  had  some  experience  in  MUM-T  operations, 
which  often  occur  when  RQ-7Bs  report  targets  to  anned  helicopters. 
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In  the  second  workshop  there  were  seven  UAS  instructor-operators,  all  senior  non¬ 
commissioned  officers.  All  were  qualified  on  medium  UAS  (e.g.  RQ-7B).  One  was  also 
qualified  on  heavy  UAS  (e.g.  MQ-1C).  All  had  operated  in  combat  zones  (i.e.,  Iraq  and  other 
areas  of  operation)  within  the  past  3  years.  All  had  experience  in  reconnaissance/security 
missions,  and  three  had  experience  in  attack  missions.  Two  had  experience  in  communications 
missions.  Most  had  some  experience  in  MUM-T  operations  in  theater. 

Materials  and  procedure.  At  the  outset  of  each  workshop,  participants  completed  an 
Informed  Consent  Form  and  then  were  given  the  Participant  Background  Survey  (Appendix  C) 
to  complete  while  the  workshop  facilitator  reviewed  the  project  objectives  and  previous  efforts 
and  explained  the  process  for  the  current  activities.  Each  of  the  workshops  was  conducted  in  two 
phases.  In  the  first  phase,  participants  produced  brief  free-text  written  responses  entered  onto 
printed  forms.  The  main  purpose  for  the  first  phase  was  to  focus  participants’  attention  on  the 
relevant  issues  and  get  them  to  organize  their  thoughts  in  order  to  promote  constructive 
discussion.  In  the  second  phase  the  participants  discussed  their  responses,  made  comparisons 
and  contrasts,  and  adjusted  their  descriptions  in  order  to  attain  a  consensus  regarding  the 
response  set. 

The  results  of  the  previous  efforts  in  the  project  were  used  to  assemble  a  list  of  the  25 
identified  MUM-T  skills  in  descending  order  of  training  criticality  (see  Table  4).  Each  skill  was 
associated  with  one  of  the  seven  task  categories  identified  in  the  Task  and  Skill  Review 
Workshop  previously  described.  Each  skill  also  was  linked  to  a  set  of  subtasks  and  perfonnance 
standards  defined  by  Army  doctrine  (see  Appendix  E).  Participants  were  presented  with  the  skill 
list  and  an  accompanying  fonn  listing  the  same  skills  with  blank  areas  for  indicators,  conditions 
and  sources.  In  the  first  phase  of  the  workshop,  participants  were  asked  to  review  the  list  and 
then  enter  responses  onto  the  blank  forms  provided.  For  the  response  fonns,  indicators  were 
defined  as  observable  behaviors  associated  with  varying  levels  of  proficiency  in  the  target  skill, 
conditions  were  defined  as  observable  events  and  processes  within  a  training  event  that  signal  the 
likelihood  of  the  target  skill  being  observed,  and  sources  were  defined  as  the  locations  (e.g. 
Portable  Institutional  Mission  Simulator)  where  an  observation  might  be  made.  As  an  example, 
the  first  skill  listed  was  Deconflict  munition  trajectories  from  airframes.  This  skill  was  rated 
first  in  criticality  (see  Table  4).  The  skill  is  associated  with  the  task  category  Call  for  and  Adjust 
Indirect  Fires  (Table  2). 

When  the  SMEs  had  completed  as  much  of  the  forms  as  needed  to  reach  a  threshold 
allowing  for  sufficient  collection  of  indicators  and  sources,  the  second  phase  of  the  workshop 
was  initiated.  In  this  phase,  the  skills  were  addressed  in  descending  order  of  training  criticality 
(see  Table  4),  and  the  group  used  the  skill  infonnation  and  their  written  responses  as  the  basis  for 
an  exchange  of  ideas  and  opinions  regarding  the  response  set.  A  workshop  facilitator  moderated 
the  session  and  took  notes  of  responses. 

Results 

The  written  responses  from  all  participants,  along  with  the  facilitator’s  notes  were 
consolidated  in  a  spreadsheet.  One  participant  entered  nothing  on  the  written  form  but 
contributed  extensively  during  the  discussion  phase.  The  spreadsheet  was  reviewed  by  the 
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research  team  to  eliminate  responses  for  indicators  that  were  deemed  unlikely  to  be  measurable 
(e.g.,  “lack  of  confidence”)  or  vague  (e.g.,  “uses  good  decision  making  ability”).  For  each  skill 
duplicate  responses  and  tangential  content  were  removed.  Responses  entered  for  indicators  were 
re-written  as  active  voice  action  statements.  Responses  for  conditions  and  sources  that  did  not 
fall  within  their  operational  definitions  were  eliminated. 

Indicators.  Indicators  are  defined  as  observable  behaviors  associated  with  varying  levels 
of  proficiency  in  the  target  skill.  Both  groups  of  participants  (i.e.,  helicopter  and  UAS)  agreed 
that  most  behavioral  indicators  of  the  skills  could  be  derived  from  the  subtask  statements  in  the 
provided  list.  Several  of  the  skills  were  identified  as  redundant.  The  helicopter  participants 
agreed  that  skills  ranked  17  and  22  through  25  (See  Table  4  for  skill  rankings)  were  all  redundant 
with  other  skills.  UAS  participants  identified  skills  ranked  17,  21  and  25  as  redundant  and 
identified  skills  ranked  22,  23  and  24  as  unimportant  with  respect  to  teaming.  As  an  example, 
skill  23,  Transmit  information  about  the  location  of  the  Unit  of  Action  had  only  one  indicator 
identified.  Know  own  position  relative  to  the  target.  This  is  a  basic  individual  skill  inasmuch  as 
UAS  and  target  coordinates  are  shown  on  the  operator’s  displays.  Skills  ranked  19,  24  and  25 
(all  related  to  describing  and  reporting  targets)  were  identified  by  helicopter  participants  as  a 
“relics  from  ISR.”  These  skills  were  considered  to  have  been  relevant  when  the  emphasis  for 
UAS  operations  was  on  ISR  missions  with  an  MI  perspective  but  are  much  less  so  for  SR 
missions  in  the  Aviation  branch.  The  UAS  participants  identified  skill  19  as  redundant  with  skill 
9,  Prioritize  the  engagement  of  targets.  Skills  ranked  17  and  22  through  25,  found  to  be 
redundant  and/or  unimportant  by  both  UAS  and  manned  participants,  were  excluded  from  further 
analysis. 

The  helicopter  pilot  participants  identified  four  skills  ranked  1,3,7  and  10  (see  Appendix 
E)  as  mostly  or  entirely  composed  of  situational  awareness  (SA).  Although  assignment  of  such  a 
label  may  appear  to  be  a  neatly  packaged  statement,  the  issues  related  to  measurement  of  SA  are 
many.  Not  the  least  of  these  is  selection  of  an  operational  definition.  The  participants  from  both 
communities  provided  other  more  specific  infonnation  for  all  of  these  skills. 

Conditions.  These  are  observable  events  and  processes  within  a  training  event  that 
signal  the  likelihood  of  the  target  skill  being  observed.  Participants  tended  to  confuse  the 
conditions  requested  on  the  response  fonn  with  conditions  and  standards  statements  relating  to 
training  tasks.  This  difference  was  clarified  by  the  moderator.  The  conditions  responses  are  not 
unique  across  skills.  Given  the  fact  that  skills  may  be  associated  with  multiple  task  categories 
and  the  fact  that  conditions  are  necessarily  fairly  broad  (i.e.,  the  scope  of  this  project  does  not 
afford  specific  analysis  of  training  environments  and  their  measurement  capabilities.),  multiple 
representation  of  conditions  across  skills  was  expected. 

Sources.  A  source  is  a  location  where  an  observation  might  be  made.  There  are  a  small 
number  of  venues  in  which  MUM-T  skills  (or  rather  the  behavioral  indicators  for  them)  may  be 
observed.  Observations  are  most  likely  to  occur  in  institutional  and  unit  training  situations  at 
crew  and  team  levels.  By  definition,  MUM-T  skills  cannot  be  observed  in  individual  training 
venues.  The  indicators  may  be  observed  in  any  of  the  usual  instructional  delivery  environments 
such  as  classroom  instruction,  and  in  constructive,  virtual  and  live  simulations.  Indicators  are 
also  observable  in  combat  operations  although  the  likelihood  that  observations  would  be 
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recorded  for  training  applications  is  low.  At  the  level  of  specificity  possible  within  the  scope  of 
this  project,  the  set  of  sources  for  benchmark  observations  in  MUM-T  is  limited.  Not 
surprisingly,  the  participants’  responses  reflected  this  limitation.  The  range  of  responses  was 
small  but  appeared  to  cover  the  possibilities  available  in  training.  The  sources,  naturally,  are 
represented  in  most  of  the  skills. 

Summary.  The  reduced  set  of  20  MUM-T  skills  yielded  a  total  of  140  indicators,  41 
conditions,  and  80  sources.  The  indicators  were  not  necessarily  unique  to  a  given  skill,  but  each 
of  the  conditions  and  sources  were.  An  example  of  a  MUM-T  skill  and  associated  indicators, 
conditions,  and  sources  is  given  in  Table  5  for  the  skill  ranked  highest  in  training  criticality.  The 
full  set  of  MUM-T  skills  and  indicators,  conditions  and  sources  appears  in  Appendix  F,  which  is 
arranged  in  descending  rank  order  of  TCI  scores  (i.e.,  training  criticality).  The  number  of 
indicators,  conditions  and  sources  declines  with  the  rank  order  of  each  skill.  This  may  be  due  in 
part  to  fatigue  of  the  participants  later  in  the  workshop  or  decreased  criticality  of  those  skills 
which  are  well  trained. 

Table  5 

Benchmark  Indicators,  Conditions,  and  Sources  for  “Deconflict  Munition  Trajectories  from 
Airframes”  Manned-Unmanned  Teaming  Skill 


Deconflict  munition  trajectories  from  airframes 
Indicators 

Call  for  fire  is  complete  and  accurate 

Operator  is  aware  that  deconfliction  is  taking  place 

Coordinates  with  Air  Traffic  Control  for  deconfliction 

Identifies  intersecting  munition  trajectories;  recommends  lateral,  vertical,  or  sequential  maneuver 

Operator  is  aware  of  positions  of  friendly  aviation  assets 

Responds  to  instructions  to  deconflict.  Moves  aircraft  to  safe  area 

Confirms  when  clear 

All  aircraft  are  in  assigned  locations 

Air  Mission  Commander  has  to  redirect  the  UAS  from  munition  trajectory  or  impact  area 
Air  Mission  Commander  has  to  call  a  cease  fire  due  to  UAS  location 
Determines  if  rounds  are  accurately  placed  on  target 
Appropriate  graphics  are  displayed  and  current 
Conditions 

Call  for  Fire  missions;  own  or  other  unit 
Troop  suppression 
Show  of  force 

Preplanned  coordinated  attack 
Target  of  opportunity  with  priority  of  fires  assumed 
Sources 

Virtual  simulations;  Aviation  Combined  Arms  Tactical  Trainer,  Portable  Institutional  Modular  Simulator 
Live  simulations 
Gunnery  training 
Combat 
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There  were  also  some  notable  open-ended  comments  made  by  SMEs  during  this 
workshop  session  for  five  skills  varying  in  training  criticality.  These  comments  are  listed  in 
Appendix  G.  In  brief,  the  most  numerous  and  important  comments  were  those  regarding  the  two 
skills  ranked  1  and  3  on  the  TCI  (i.e.,  Deconflict  munition  trajectories  from  airframes  and 
Transmit  information  about  method  of  attack.).  Comments  regarding  these  MUM-T  skills 
reflected  concern  for  a  need  for  joint  mission  planning  by  UAS  and  manned  elements,  and  the 
lack  of  adequate  communication  and  coordination,  as  well  as  a  need  to  train  both  UAS  and 
manned  aircrews  in  the  deconfliction  of  two  aircraft  with  very  different  operational  envelopes. 
The  proper  routing  of  communications  and  the  lack  of  shared  Tactical  Standard  Operating 
Procedures  appeared  to  be  another  major  concern. 

Discussion  and  Recommendations 

The  need  to  more  centrally  integrate  UAS  into  aviation  operations  has  resulted  in 
increased  MUM-T  during  aviation  SR  missions.  As  a  consequence,  greater  emphasis  on  MUM- 
T  and  SR  mission  skills  will  occur  in  UAS  training,  particularly  for  RQ-7B  operators.  However, 
MUM-T  skills  are  relatively  new  to  both  helicopter  aircrews  and  UAS  operators,  and  the 
doctrine  for  MUM-T  is  still  evolving.  In  order  to  develop  effective  MUM-T  training,  it  is  crucial 
to  understand  the  specific  skills  required  for  MUM-T  task  performance  and  the  most  efficient 
ways  to  train  UAS  operators  on  MUM-T  skills.  Therefore,  the  goal  of  the  research  reported  here 
was  to  specify  an  empirically-derived  set  of  MUM-T  critical  skills  and  to  develop  benchmark 
indicators  for  perfonnance  on  those  skills. 

In  order  to  meet  this  goal,  this  research  used  a  combination  of  doctrinal  review  and  SME 
input  to  (a)  identify  candidate  MUM-T  skills,  (b)  determine  the  training-criticality  of  the  MUM- 
T  skills,  (c)  revise  the  set  of  desired  skills,  and  (d)  produce  a  set  of  benchmark  indicators  for  each 
training-critical  MUM-T  skill.  Altogether,  25  MUM-T  skills  were  identified.  Of  those  25  skills, 
20  MUM-T  skills  were  detennined  to  be  training-critical  skills.  Examples  of  training-critical 
skills  included  Deconflict  munition  trajectories  from  airframes,  Utilize  standard  execution 
commands  to  initiate  attack,  and  Conduct  call  for  direct  fires. 

The  evolutionary  state  of  MUM-T  doctrine  and  tactics  presented  a  challenge  to  the 
identification  of  MUM-T  Skills.  However,  participants  in  the  Task  and  Skill  Review  Workshop 
including  several  members  of  DOTD,  including  a  doctrine  writer  from  the  Director’s  staff,  saw 
the  draft  list  of  proposed  MUM-T  skills  as  adequate  and  within  the  ambit  of  current  doctrine. 
Minor  changes  to  the  list  were  recommended  as  some  skills  were  added  and  others  combined. 
Thus,  the  objective  of  identifying  and  validating  (as  to  doctrine)  tasks  for  MUM-T  and  their 
underlying  skills,  was  met.  One  important  thing  learned  in  conjunction  with  the  Task  and  Skill 
Review  Workshop  is  that,  in  current  DOTD  thinking,  MUM  operations  will  involve  peer-to-peer 
using  voice-only  communications.  Unlike  previous  UAS  operations,  communications  in  most 
cases  will  not  use  the  Tactical  Operations  Center  as  an  intermediary. 

Another  challenge  to  the  identification  of  MUM-T  skills  was  that  many  of  the  helicopter 
pilots  and  UAS  operators  participating  in  the  SME  workshops  were  not  fully  aware  of  the 
capabilities  of  their  counterpart  personnel  and  airframes.  That  is,  for  example,  many  scout-attack 
pilots  did  not  know  the  extent  or  nature  of  training  UAS  operators  receive.  Each  group  of 
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participants  (i.e.,  pilots  and  UAS  operators)  tended  to  defer  to  the  other  when  issues  of  specific 
airframes  arose.  As  a  consequence,  the  input  on  some  skills  is  biased  toward  one  perspective 
instead  of  representing  an  integrated  perspective.  The  fact  that  the  two  groups  of  participants 
were  not  fully  aware  of  their  counterparts’  capabilities  also  indicated  that,  despite  an  expectation 
of  burgeoning  reliance  on  MUM-T  in  the  near  term,  manned  and  unmanned  communities  are  still 
insufficiently  integrated.  For  a  more  complete  discussion  of  these  differential  perspectives,  the 
reader  is  referred  to  Stewart,  et  al.  (2012). 

The  training  criticality  of  MUM-T  skills  was  based  on  perceived  current  proficiency  and 
on  the  consequences  of  team  performance.  More  specifically,  MUM-T  skills  were  defined  as 
training-critical  if  (a)  UAS  operators  who  have  completed  AIT  cannot  perform  them,  (b)  the 
skills  are  essential  for  successful  completion  of  missions,  and  (c)  the  lack  of  proficiency  at  the 
skills  can  endanger  personnel  and  equipment.  These  dimensions  were  captured  and  fonnalized 
in  the  TCI.  The  prioritization  of  skills  on  the  composite  TCI  indicated  that  the  most  critical  skills 
were  related  to  tactical  SR  operations,  whereas  the  least  critical  skills  were  traditional  ISR  skills. 
There  was  little  variation  across  skills  in  ratings  of  importance  to  missions  or  ratings  of 
importance  to  personnel  and  equipment.  Indeed,  some  of  the  skills  rated  as  most  important  were 
ISR  skills  that  are  also  proficiently  performed  by  newly-trained  UAS  operators.  Consequently,  it 
seems  that  the  determinant  of  training  criticality  was  the  perceived  ability  of  UAS  operators 
graduating  from  AIT  to  effectively  perform  the  identified  skills.  One  must  note  that  was  one 
application  of  the  TCI  to  a  set  of  skills  all  of  which  were  judged  to  be  at  least  moderately 
important.  Therefore  variation  due  to  importance  was  low  relative  to  variation  due  to  estimated 
performance.  For  this  reason,  it  would  be  premature  to  state  that  estimated  performance  is  the 
most  essential  dimension,  and  the  three  importance  dimensions  are  redundant.  Further 
applications  of  the  TCI  to  other  skill  sets  are  needed  to  determine  the  relative  weights  of  the 
dimensions  comprising  the  TCI. 

Measuring  Performance  of  MUM-T  Skills 

In  order  to  establish  benchmarks  to  measure  MUM-T  skills,  140  indicators  (i.e., 
observable  behaviors  associated  with  levels  of  proficiency  for  a  given  skill)  were  developed 
across  the  final  set  of  20  MUM-T  skills.  The  benchmarks  also  required  the  determination  of  41 
unique  conditions  (i.e.,  events  signaling  the  likelihood  that  a  given  skill  can  be  observed)  and  80 
unique  sources  (i.e.,  locations  where  the  observation  can  be  made)  for  the  MUM-T  skill  set.  The 
indicators,  conditions,  and  sources  are  prototypical  in  the  sense  that  they  are  currently  not 
systematically  used  to  assess  performance  in  MUM-T  operations.  It  should  be  emphasized  that 
extensive  field  testing  and  refinement  are  necessary  before  these  prototype  tools  can  become 
usable  by  trainers  and  evaluators.  It  is  likely  that  many  of  the  indicators  could  be  used  in 
structured  training  environments,  but  this  will  require  detailed  assessment  of  the  data  recording 
capabilities  embedded  in  the  simulations  and  system  equipment  as  well  as  assessment  of  the 
capacity  for  accommodation  of  trained  observers.  Perhaps  the  most  usable  assessment  tools  in 
the  near  term  would  be  behaviorally-anchored,  observer-based  measures. 

In  order  to  develop  specific  measures  of  MUM-T  skill  from  the  developed  benchmarks, 
refinement  of  the  indicators,  conditions,  and  sources  will  be  necessary.  The  available  training 
delivery  environments  should  be  reviewed  to  determine  more  specifically  the  sources  where 
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MUM-T  performance  indicators  may  be  observable.  This  review  should  include  classroom 
training  as  well  as  constructive,  virtual,  and  live  training  at  institutions  and  homes  stations.  Each 
of  the  training  environments  should  be  analyzed  for  content  (e.g.  scenarios)  that  contain  the 
conditions  under  which  the  indicators  may  be  observable.  The  analyses  should  be  aimed  at 
identification  and  functional  description  of  data  elements  relevant  to  the  Indicators.  The  analyses 
could  be  done  through  SME  observation  (e.g.,  McGilvary,  Leibrecht,  &  Lockaby,  2008;  Sticha, 
Weaver,  Ford,  &  Campbell,  2011)  or  through  automation  in  constructive  or  virtual  simulations 
(Dorsey,  et  ah,  2009).  One  example  approach  to  these  analyses  is  Targeted  Acceptable  Response 
to  Generated  Events  or  Tasks  (TARGETs)  (Dwyer,  Fowlkes,  Oser,  Salas,  &  Lane,  1997). 
TARGETS  is  an  event  based  training  development  and  measurement  method  that  identifies 
events  within  a  scenario  that  trigger  the  production  of  indicator  behaviors  for  observation.  ARI 
is  currently  in  the  process  of  developing  both  observer-based  and  system-based  (i.e.,  automated) 
perfonnance  measures  for  aviation  collective  training  (Seibert,  Diedrich,  Stewart,  Bink  & 
Zeidman,  2011).  Although  these  measures  were  developed  prior  to  the  stand-up  of  FSCAB 
units,  it  is  likely  that  this  measurement  technology  can  be  adapted  to  MUM-T  training. 

The  development  of  behavioral  measures  from  the  indicators  should  take  into  account  the 
intra-  and  interpersonal  processes  involved  in  team  operations.  There  is  a  considerable  literature 
on  team  processes  and  consideration  of  this  body  of  knowledge  will  provide  direction  for  the 
structure  and  development  of  MUM-T  measures.  For  example,  cross-training  of  system  operator 
roles  is  universal  within  Army  UAS  operators  and  within  helicopter  pilots  but  is  not  practiced 
between  MUM-T  elements.  Cross-training  within  MUM-T  is  likely  to  increase  the  extent  and 
quality  of  a  shared  mental  model  for  MUM-T.  Army  Aviation  is  aware  of  the  need  for  this 
training,  and  is  currently  experimenting  with  various  training  events  intended  to  make  the  scout- 
attack  pilot  more  sensitive  to  the  role  of  the  UAS  operator. 

Training  MUM-T  Skills 

As  indicated  by  the  training  criticality  assessments,  incorporation  of  UAS  into  SR 
missions  has  added  the  requirements  for  new  skills  that  are  not  currently  trained  as  part  of  UAS 
AIT.  These  new  skills  require  a  more  active  role  for  the  UAS  operator  in  various  aspects  of  the 
missions.  The  general  term:  developing  the  situation  is  often  employed  to  represent  many  of  the 
target  identification,  attribution  of  target  intent,  communicating  and  reporting  behaviors 
delineated  in  the  extensive  list  of  indicators  (see  Appendix  F).  Developing  the  situation  will 
require  extensive  new  learning  of  cognitive  and  procedural  skills  on  an  individual  level  in  order 
for  the  UAS  crew  to  become  more  effective  in  MUM-T  missions.  The  MUM-T  skills  and 
corresponding  benchmarks  obtained  in  the  present  research  effort,  though  prototypes,  may  prove 
beneficial  as  guides  to  detennining  and  prioritizing  those  cognitive  and  procedural  skills  that 
must  be  trained.  UAS  crews  must  leam  procedures  and  techniques  currently  not  taught  at  AIT 
such  as  laser  designation  of  targets  for  armed  helicopters  as  well  as  the  exclusion  zones  and  other 
characteristics  of  laser  guided  munitions.  The  challenge  for  Anny  Aviation  is  to  detennine 
where  and  how  these  new  skills  are  to  be  trained. 

Those  skills  most  in  need  of  training  were  SR  skills,  the  same  as  those  perfonned  by 
crews  of  manned  scout  helicopters,  and  for  which  the  majority  of  UAS  operators  were  perceived 
by  SMEs  as  lacking  sufficient  background  and  experience.  Obviously,  if  they  are  to  become 
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effective  MUM  team  members,  UAS  operators  must  master  these  skills.  There  are  many 
questions  as  yet  unanswered,  as  to  how  much  training  time  will  be  required  for  mastery,  by 
Soldiers  who  differ  on  many  background  dimensions  from  aviators  (e.g.,  rank,  method  of 
selection,  flight  aptitude).  Thus  another,  related  challenge  is  if  these  skills  can  be  acquired  and 
sustained  to  the  same  level  of  proficiency  as  for  the  crews  of  manned  aircraft. 

Thus,  the  training  aspect  of  MUM-T  is  crucial,  not  only  for  UAS  operators,  but  also  for 
pilots  who  may  not  fully  understand  the  capabilities  and  limitations  of  the  RQ-7B.  For  example, 
RQ-7B  usually  operates  at  an  altitude  of  6,000-8,000  ft,  whereas  scout  and  attack  helicopters 
typically  do  not  operate  above  1,000  ft.  Consequently,  the  operational  environments  of  the  two 
aircraft  types  are  very  different  and  must  be  mutually  understood  by  both  team  members.  In 
addition,  because  the  RQ-7B  has  substantially  greater  endurance  than  a  manned  helicopter  such 
as  the  AH-64D,  the  UAS  crew  will  often  be  required  to  take  initiative  in  identifying  and 
reporting  a  target  to  an  AH-64D  that  is  joining  the  UAS  to  operate  as  a  part  of  a  MUM-T 
operation. 

Live  training  is  available  to  UAS  units  at  home  station  but  is  limited  in  the  amount  of 
participation  by  other  assets.  Formation  of  MUM-T  in  home  station  live  training  will  generally 
require  that  helicopter  elements  be  filled  by  role  players.  Live  training  in  large  federations,  such 
as  Combat  Training  Centers,  are  only  rarely  available  and  provide  a  greatly  curtailed  maneuver 
box  for  aviation  assets  and  limited  ability  to  incorporate  UAS  into  SR  missions  (Stewart,  Barker, 
&  Bilik,  2010).  Likewise,  inclusion  of  UAS  in  large-scale  virtual  training  exercises  such  as  the 
Aviation  Training  Exercise  (ATX)  has  been  minimal  to  date.  Typically,  UAS  crew  positions 
during  ATX  have  been  filled  by  role-players  (e.g.,  aviation  warrant  officers  on  casual  duty). 
However,  inclusion  of  UAS  personnel  in  ATX  is  expected  to  be  more  frequent  as  more  FSCABs 
are  organized  and  come  to  participate  in  ATX  staring  in  the  summer  of  2012.  Asa  result,  ATX 
will  become  a  primary  training  environment  for  MUM-T. 

Conclusions 

The  TCI  composite  index,  similar  in  many  ways  to  the  Air  Force’s  MEC  methodology, 
was  successfully  exercised  in  the  current  research  effort  to  identify  and  prioritize  training-critical 
MUM-T  skills.  Both  of  these  methodologies  promise  to  be  useful  tools  for  pinpointing 
specifically  what  skills  require  priority  attention  for  training  when  new  systems,  and/or  new 
doctrines  and  tactics,  are  under  development.  This  report  presents  a  snapshot  of  MUM-T  skills 
and  benchmark  performance  indicators  based  on  the  current  UAS  capabilities  and  the  capabilities 
envisioned  for  the  near  future.  However,  there  are  numerous  changes  taking  place  in  the  Anny’s 
UAS  assets  and  their  employment.  These  changes  are  taking  place  in  organization,  in  missions, 
in  airframes,  in  mission  equipment,  and  in  tactics,  techniques  and  procedures.  The 
organizational  changes  are  driving  the  shift  in  emphasis  from  ISR  toward  SR  at  a  greater  than 
planned  rate.  This  shift  is  driving  an  expansion  in  the  breadth  of  missions  in  which  they  are 
employed.  This,  then,  drives  the  development  of  new  TTP.  At  the  same  time,  changes  to  the 
airframes  and  their  mission  equipment  packages  are  adding  capabilities  which  feed  further 
expansion  of  missions  and  TTP.  A  major  organizational  change,  beyond  the  transfer  from  MI  to 
Aviation,  is  the  development  of  FSCABs  (Stewart  et  ah,  2012).  Employment  of  UAS  in  attack 
and  Signal  Intelligence  missions  is  increasing. 
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At  the  current  stage  of  MUM-T  doctrine  and  training  development,  several  key 
challenges  exist  to  the  successful  implementation  of  efficient  and  effective  instructional 
strategies.  These  have  been  addressed  at  length  in  the  current  paper,  but  should  be  summarized 
briefly  here.  First,  it  is  evident  that  UAS  operators  must  acquire  and  master  many  of  the  SR 
skills  that  until  recently  were  performed  solely  by  aircrews  of  armed  scout  and  attack  helicopters. 
ARI  research  (Stewart,  et  ah,  2011;  Stewart,  et  ah,  2012)  has  indicated  that  training  in  AIT  still 
emphasizes  on  ISR  as  opposed  to  SR  operations.  Secondly,  the  onus  of  training  critical  MUM-T 
skills  rests  with  the  operational  units,  since  AIT  does  not  provide  this  training.  A  challenge  is 
the  lack  of  opportunity  to  acquire  and  maintain  MUM-T  skills  at  home  station  in  live  training, 
thus  necessitating  the  use  of  virtual  training  environments,  including  purpose-built  simulators 
and  portable  training  devices  at  home  station,  and  participation  in  ATX  exercises.  Third, 
performance  measurement  technology  lags  behind  the  state  of  the  art  for  simulation  for 
unmanned  as  it  also  does  for  manned  aviation,  and  this  technology  is  crucial  for  efficient  and 
effective  unit  level  training.  Such  measures  will  be  required  for  MUM-T. 

Finally,  but  nonetheless  important,  is  the  need  to  integrate  training  for  UAS  and  manned 
aircrews,  starting  at  the  Combat  Skills  Phase  of  Initial  Entry  Rotary  Wing  training,  This  would 
involve  mission  planning  exercises  for  OH-58D  and  AH-64D  aircrews,  in  which  RQ-7B  crew 
participated  directly  in  the  planning  and  rehearsal  of  simulated  SR  missions.  The  result  would  be 
assimilation  of  the  UAS  aircrews  into  the  mainstream  of  Anny  Aviation  training.  MUM-T  must 
not  focus  entirely  on  the  training  needs  of  the  UAS  operators;  pilots  of  helicopters  such  as  the 
OH-58D  must  be  familiarized  with  the  capabilities  and  especially  limitations  of  UAS.  The  same 
requirement  should  apply  to  BCT  leadership,  which  may  not  be  familiar  with  means  of 
employing  RQ-7B  as  a  tactical  asset.  So  not  only  is  training  of  aircrews  important,  but  education 
of  leaders  as  well. 

As  doctrine  specific  to  the  Scout/ Attack  and  UAS  communities  becomes  more  closely 
coordinated,  differences  between  the  communities  in  perspective  and  practice  will  be  reduced. 
This  coordination  will  simplify  MUM-T  training.  Communication  is  one  of  the  principal  factors 
in  MUM-T  operations  and  is  an  area  in  which  consistent  doctrine  will  have  a  large  impact. 
Mission  planning  is  another  area  in  which  the  development  of  doctrine  consistent  with  MUM-T 
will  be  beneficial.  Because  of  the  disparity  in  endurance  a  UAS  crew  may  team  with  three  or 
four  rotary  wing  missions  in  a  single  flight.  This  tempo  could  place  the  UAS  flight  inside  the 
mission  planning  cycle  of  the  helicopter  team  elements.  Therefore,  the  UAS  crew  could  be 
placed  in  the  position  of  planning  missions  while  operating  their  system.  New  procedures  may 
be  necessary  to  ensure  that  the  UAS  element’s  planning  is  done  completely  without  adversely 
affecting  their  workload.  Mission  planning  and  team  training  are  also  affected  by  geographical 
separation  of  the  helicopter  and  UAS  elements. 

Finally,  it  should  be  noted  that  many  of  the  critical  skills  identified  and  prioritized  in  the 
present  research  can  be  considered  platfonn-independent.  Indeed,  SR  operations  are  based  upon 
time-honored  Cavalry  scouting  doctrine  and  tactics  that  were  in  use  prior  to  the  invention  of  the 
aircraft.  Also,  reconnaissance  and  scout  activities  have  long  been  a  part  of  naval  operations. 
Thus,  many  of  these  skills  would  be  applicable  to  a  new  generation  of  ground-based  and  even 
marine  unmanned  reconnaissance  systems  that  perform  the  role  previously  performed  by  the 
venerable  Anny  scout  and  pathfinder  teams. 
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Preliminary  findings  from  the  present  research  effort,  along  with  recommendations,  have 
been  disseminated  to  the  UAS  and  Scout- Attack  communities,  through  joint  briefings  to  TCM- 
Reconnaissance-Attack  and  TCM-UAS. 
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Acronyms 


AIT 

ARI 

ATX 

Advanced  Individual  Training 

U.S.  Army  Research  Institute  for  the  Behavioral  Sciences 
Aviation  Training  Exercise 

BCT 

Brigade  Combat  Team 

DOTD 

Directorate  of  Training  and  Doctrine 

FSCAB 

Full  Spectrum  Combat  Aviation  Brigade 

ISR 

Intelligence,  Surveillance,  and  Reconnaissance 

MEC 

Mission  Essential  Competencies 

MI 

MUM-T 

Military  Intelligence 

Manned-Unmanned  Teaming 

SA 

SME 

SR 

Situational  Awareness 

Subject-Matter  Expert 

Scout-Reconnaissance 

TARGETS 

TCI 

TCM-UAS 

TTP 

Targeted  Acceptable  Response  to  Generated  Events  or  Tasks 
Training  Criticality  Index 

Training  and  Doctrine  Command  Capabilities  Manager 
Tactics,  Techniques,  and  Procedures 

UAS 

Unmanned  Aircraft  System 
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APPENDIX  A 

ARMY  AVIATION  MISSIONS  INVOLVING  MANNED-UNMANNED  TEAMING 
AND  PRIMARY  REFERENCE  SOURCE 


A-l 


Army  Aviation  Missions  Involving  Manned-Unmanned  Teaming  and  Primary  Technical 
Reference 


Army  Aviation  Mission 

Primary  Reference  Source 

Conduct  Raids 

FM  3-90,  Tactics 

Provide  Aerial  Escort  and  Suppressive  Fires 

FM  3-04.111,  Aviation  Brigades 

Provide  Mobile  Firepower 

FM  3-04.111,  Aviation  Brigades 

Destroy  Enemy  Formations 

FM  3-04.111,  Aviation  Brigades 

Conduct  Limited  Joint  Suppression  of  Enemy  Air  Defenses 
Operations 

FM  3-04.111,  Aviation  Brigades 

Coordinate  and  Adjust  Indirect  Fires 

FM  3-04.111,  Aviation  Brigades 

Conduct  Joint  Air  Attack  Team  with  Close  Air  Support  and  Field 
Artillery 

FM  3-04.111,  Aviation  Brigades 

Conduct  Air  Assaults 

FM  3-04.111,  Aviation  Brigades 

Support  Maneuver  Forces  Through  Close  Combat  Attack 

FM  3-04.111,  Aviation  Brigades 

Conduct  Area  Reconnaissance 

FM  3-04.126,  ARH  Operations 

Conduct  Zone  Reconnaissance 

FM  3-04.126,  ARH  Operations 

Conduct  Route  Reconnaissance 

FM  3-04.126,  ARH  Operations 

Conduct  Area  Security  Operations 

FM  3-04.126,  ARH  Operations 

Conduct  Screening  Operations 

FM  3-04.126,  ARH  Operations 

Conduct  Guard  and  Covering  Force  Operations 

FM  3-04.126,  ARH  Operations 

Conduct  Chemical,  Biological,  Radiological,  and  Nuclear 
Reconnaissance 

FM  3-04.111,  Aviation  Brigades 

Conduct  Aeromedical  Evacuation 

FM  3-04.111,  Aviation  Brigades 

Conduct  Downed  Aircraft  Recovery  Team  Operations 

FM  3-04.513,  Aircraft  Recovery  Operations 

Participate  in  Personnel  Recovery  Operations 

FM  3-50.1,  Army  Personnel  Recovery 

Conduct  Command  and  Control  Operations 

FM  3-04.111,  Aviation  Brigades 

Conduct  Aerial  Sustainment  Operations 

FM  3-04.111,  Aviation  Brigades 

Notes'.  ARH  =  Attack  Reconnaissance  Helicopter;  FM  =  Field  Manual. 


A-2 


APPENDIX  B 

TASK  AND  SKILL  REVIEW  WORKSHOP  PROTOCOL 


B-l 


Task  and  Skill  Review  Workshop  Protocol 


The  objective  of  the  review  workshop  is  to  present  the  draft  list  of  the  Aviation  unit  missions, 
Aircrew  coordination  tasks,  and  UAS  coordination  skills  to  determine  whether  the  information 
accurately  represents  the  interactive  role  of  UAS  operators  in  aircrew  teaming  scenarios.  The 
meeting  participants  will  discuss  the  questions  provided  below  to  develop  of  a  final  list  of 
manned  and  unmanned  missions,  tasks,  and  skills. 

Missions 

1 .  Are  there  aviation  unit  missions  that  are  missing  from  this  list  (e.g.,  missions  not  covered 
in  Army  doctrine)? 

2.  Are  there  aviation  unit  missions  that  should  be  removed  from  this  list  (e.g.,  mission  is  no 
longer  performed  in  the  field)? 

3.  Are  there  aviation  unit  missions  that  overlap  or  are  the  missions  independent  of  each 
other?  If  they  overlap  can  the  statements  be  combined  into  a  single  mission? 

4.  Are  the  aviation  unit  missions  listed  valid  in  terms  of  actual  practice  in  the  field? 

5.  What  future  aviation  unit  missions  are  required  for  manned  and  unmanned  teaming? 

Tasks 

6.  What  aircrew  coordination  tasks  are  required  for  the  attack  and  reconnaissance  missions? 

7.  Are  we  missing  any  tasks  from  this  list  (e.g.,  tasks  not  covered  in  Army  doctrine)? 

8.  Are  there  tasks  that  should  be  removed  from  the  list  (e.g.,  task  does  not  involve  aircrew 

coordination)? 

9.  Do  the  task  statements  overlap  or  are  they  independent  of  each  other?  If  they  overlap  can 
the  statements  be  combined  into  a  single  task? 

10.  Are  the  task  statements  listed  valid  in  terms  of  actual  practice  in  the  field? 

1 1 .  What  future  tasks  will  UAS  operators  need  to  perfonn  in  aircrew  coordination  teaming? 

Skills 

12.  What  coordination  skills  do  UAS  operators  need  to  perform  the  tasks? 

13.  Are  we  missing  any  skills  from  this  list  (e.g.,  skills  not  covered  in  Army  doctrine)? 

14.  Are  there  skills  that  should  be  removed  from  the  list  (e.g.,  the  skill  does  not  aircrew 

coordination)? 

15.  Are  there  skills  that  should  be  moved  under  another  task  statement? 

16.  Do  the  skills  overlap  or  are  they  independent  of  each  other?  If  they  overlap  can  the 
statements  be  combined  into  a  single  skill? 

17.  What  prerequisites  are  needed  prior  to  training  the  skills  listed? 

18.  What  type  of  training  is  currently  provided  for  the  skills  listed? 

19.  Are  the  skills  listed  valid  in  terms  of  actual  practice  in  the  field? 

20.  What  are  the  sequential  perfonnance  dependencies  between  the  skills? 
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Manned-Unmanned  Teaming  Skill  Review  Workshop 

_ Participant  Background  Form _ 

1 .  What  is  your  rank? 


2.  What  is  your  branch  assignment? 

□  Military  Intelligence 

□  Aviation 

□  Other _ 

3.  When  was  your  last  tour  operating  in  a  combat  zone? 

□  Within  the  past  three  years 

□  Four  or  more  years  ago 

□  I  have  not  operated  in  a  combat  zone 

4.  What  type  of  aircraft  did  you  primarily  operate  during  your  last  tour? 

□  Attack  Helicopter 

□  Observation  Helicopter 

□  Other  Helicopter 

□  Heavy  UAS  (e.g.,  Predator,  Sky  Warrior) 

□  Medium  UAS  (e.g.  Shadow,  Hunter) 

□  Other  UAS 

□  None  of  the  above  (specify  the  system) _ 

5.  Mark  the  system(s)  you  are  currently  qualified  to  operate: 

□  OH-58D 

□  AH-64D 

□  Other  Helicopter  Systems 

□  Heavy  UAS  (e.g.,  Predator,  Sky  Warrior) 

□  Medium  UAS  (e.g.  Shadow,  Hunter) 

□  Other  UAS  system _ 

□  None  of  the  above  (specify  the  system) _ 

6.  Mark  the  aviation  unit  missions  you  have  experience  with: 

□  Attack 

□  Reconnaissance/Security 

□  Communications  Relay 

□  Other  (please  specify) _ 

7.  Indicate  the  area(s)  of  operations  you  have  experience  with: 
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Manned-Unmanned  Teaming  Skills  Reference  Information 

Skill 

Reference  and  Page  Number 

FM  3-04.155 

FM  34-212. 

TC  1-600. 

FM  3-04.126 

FM  6-99.2 

Conduct  call  for  direct  fires 

3-17 

Conduct  call  for  indirect  fires 

A-62 

Conduct  target  handover 

3-36 

Deconflict  munition  trajectories  from  airframes 

B-ll 

Develop  and  send  COP  information  to  air-ground  team 

3-14 

Find/track  targets 

3-28 

Gain  and  Maintain  Enemy  Contact 

3-2 

Perform  BDA 

3-40 

Prioritize  the  engagement  of  targets 

3-27 

Provide  accurate  description  of  the  target  to  support  selection 

3-36 

Provide  confirmation  of  the  target  prior  to  engagement 

3-36 

Provide  early  warnings,  ambush  detection,  overwatch,  threat  identification 

3-4 

Provide  target  description  information 

3-36 

Provide  the  direction  of  the  target  in  degrees  and  range  from  the  battle  position 

3-36 

Provide  the  target  location 

3-36 

Select  the  best  weapon  systems  to  engage  the  target 

3-27 

Switch  roles  of  laser  designator  and  missile  launch  platforms 

3-37 

Transmit  imagery,  sensor  data,  tactical  situational  maps,  overlays,  and  automated 
reports 

3-3 

Transmit  information  about  the  location  and  direction  of  the  UA  as  it  relates  to  the 
target 

3-37 

Transmit  information  about  the  location  of  threat  forces,  terrain,  and  obstacles  that 
influence  operations 

3-3 

Transmit  information  about  the  method  attack 

3-36 

Utilize  joint,  Army,  and  civilian  personnel  recovery  terminology 

Utilize  radio  communication  and  signal  operating  procedures 

Utilize  standardized  execution  commands  to  initiate  attack 

Utilize  standardized  report  fonnats 

3-36 

4-6 

3-89 
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Criticality 

Rank 

Skill 

Task 

Category 

Sub-Tasks 

Standard 

1 

Deconflict  munition 
trajectories  from 
airframes 

Call  for  and 
Adjust  Indirect 
Fires 

•  Determine  doctrinal,  regulatory,  or 
TACSOP  requirements  for  munitions  clearance 
vice  aircraft  locations. 

•  Visualize  where  friendly  fires  will  be 
coming  from  in  relation  to  their  ejected 
targets  and  friendly  aircraft  locations. 

•  Confine  the  aircraft  to  a  specific  region. 

•  Utilize  the  common  grid  reference  system 

•  Notify  appropriate  contact  when 
potential,  or  actual,  conflicts  are  detected. 

•  Aircraft  are  outside  the  minimum  safety 
zones  for  friendly  direct  and  indirect  fire 
trajectories  and  their  inpact  area  when  the  fire 
occurs,  according  to  the  type  of  weapons  and 
munitions  being  fired. 

2 

Utilize  standardized 

execution  commands  to 

initiate  attack 

Share  Targeting 
Information 

•  Determine  the  execution  commands 
required  for  the  type  of  attack. 

•  Issue  command  IAW  TACSOP  and 
appropriate  doctrine. 

•  Ensure  information  complies  with  Amiy 
infonnation  standards. 

•  Shared  tactical  information  with  manned 
aircraft  on  the  method  of  attack  using  anny 
aviation  concepts,  terms  and  graphics, 
containing  all  the  required  elements  of  the 

FRA  GO  or  attack  order. 

3 

Transmit  information 

about  the  method 
attack  (i.e.,  scheme  of 
maneuver,  fire 
distribution,  and 

maneuver  for  the 
attack) 

Share  Targeting 
Information 

•  Determine  elements  required  to  share 
concerning  the  method  of  attacking  the  target. 

•  Prepare  an  appropriate  communications  to 
share  the  infonnation  (FRA GO,  call  for  fire,  fire 
command). 

•  Transmit  infonnation. 

•  Ensure  infonnation  complies  with  Anny 
infonnation  standards. 

•  Shared  tactical  infonnation  with  manned 
aircraft  on  the  method  of  attack  using  anny 
aviation  concepts,  terms  and  graphics, 
containing  all  the  required  elements  of  the 

FRA  GO  or  attack  order. 

4 

Switch  roles  of  laser 
designator  and  missile 
launch  platforms 

Conduct 

Cooperative 

Engagements 

•  Detennine  other  party  to  transfer  of  roles. 

•  Contact  party  to  verify  procedures  to  be 
used. 

•  Assess  the  target  location,  laser 
designator  position,  and  UA  location. 

•  Effect  the  transfer. 

•  Test  the  transfer. 

•  UAS  transferred  duties  of  designator  or 
missile  launch  enabling  a  timely  and  effective 
delivery  of  munitions . 

Criticality 

Rank 

Skill 

Task 

Category 

Sub-Tasks 

Standard 

5 

Conduct  call  for  direct 

fires 

Conduct 

Cooperative 

Engagements 

•  Collect  information  required  to  call  for 
direct  fire. 

•  Determine  requirements  for  a  clearance  of 
this  type  of  fire. 

•  Determine  who  may  be  available  to 
provide  fire. 

•  Deliver  a  fire  request  or  fire  command. 

•  Prepare  to  support  the  execution  of  the 
fire  mission  (e.g.,  spotting,  covering 
movement  into  position,  watching  flanks 
during  execution). 

•  Provide  infonnal  BDA  to  firing  unit. 

•  Request  for  direct  fire  was  tactically 
sound  and  rapidly  understood,  and  was  IAW 
TACSOP  and  current  mles  of  engagement. 

6 

Select  the  best  weapon 
systems  to  engage  the 
target  (e.g., 
lethal/ nonlethal, 
munitions  effect, 
collateral  damage 
assessment) 

Conduct 

Cooperative 

Engagements 

•  Determine  target  effects  requirements . 

•  Determine  risk  for  collateral  damage. 

•  Determine  current  mles  of  engagement  for 
this  situation. 

•  Recommend  weapons  system  and/or 
munitions  type  based  on  mission  and  risk 
assessment. 

•  Prepare  information  to  inform  clearance  of 
fires  authority. 

•  Respond  to  requests  for  more 
information. 

•  Weapons  selection  ensured  the  required 
target  effect  with  minimal  collateral  damage, 
and  IAW  current  mles  of  engagement. 

7 

Develop  and  send 
common  operating 
picture  information  to 
air-ground  team 

Communicate 

Requirements 

•  Determine  proper  method  to  post  COP- 
related  information. 

•  Maintain  fresh  COP  infonnation. 

•  Follow  the  TACSOP  for  COP  infonnation 
procedures  and  standards. 

•  Verify  with  other  parties  that  COP 
information  is  being  shared  properly. 

•  Ensure  infonnation  complies  with  Amiy 
infonnation  standards. 

•  COP -related  information  sharing  was  in 
accordance  with  tactical  communications  SOP 

•  Infonnation  was  complete,  accurate, 
reliable,  precise,  usable,  and  timely. 

E-4 


Criticality 

Rank 

Skill 

Task 

Category 

Sub-Tasks 

Standard 

8 

Utilize  joint.  Army,  and 
civilian  personnel 
recovery  terminology 

Communicate 

Requirements 

•  Maintain  personnel  recovery  operations 
references . 

•  Utilize  the  PR  terminology  from  FM  3- 
04.126. 

•  Execute  PRbattle  drill  in  accordance  with 

doctrine  and  TACSOP. 

•  Communication  was  in  accordance  with 
personnel  recovery  operations  TACSOP  and 

JP  3-50. 

9 

Prioritize  the 
engagement  of  targets 

Identify  Threats 

•  Compare  identified  target  to  target  priority 
list. 

•  Recognize  targets  that  pose  an  immediate 
danger  that  are  not  necessarily  on  the  list. 

•  Recognize  targets  that  may  have  difficulty 
being  engaged  by  organic  weapons  systems. 

•  Prioritize  actions  IAW  preceding 
considerations. 

•  Report  target  priority  and 
recommendations . 

•  Ensure  information  complies  with  Army 
information  standards. 

Prioritized  identified  enemy  targets  based  on: 

•  Target  list 

•  Target  threat 

•  Ability  to  engage  target  with  available 

weapons 

10 

Gain  and  Maintain 
Enemy  Contact 

Conduct  Aerial 

Observation 

•  Identify  enemy. 

•  Maintain  visual  contact  with  moving 
enemy  even  when  changing  location. 

•  Provide  imagery  as  required. 

•  Acquired  the  intended  enemy  target(s). 

•  Once  acquired,  did  not  lose  contact  with 
moving  enemy  target(s). 

11 

Conduct  call  for 

indirect  fires 

Call  for  and 
Adjust  Indirect 
Fires 

•  Collect  information  required  for  a  call  for 
indirect  fire. 

•  Determine  requirements  for  a  clearance  of 
this  type  of  fire. 

•  Complete  a  fire  request. 

•  Determine  contact  for  requesting  indirect 
fires. 

•  Transmit  fire  request  and  respond  to 
requests  for  more  information. 

•  Prepare  to  support  the  execution  of  the 
fire  mission. 

•  Acknowledge  fire  mission  receipt. 

•  Call  for  indirect  fire  was  IAW  TACSOP 
and  FM  3-09.30,  enabling  a  successful  fire 
mission. 

Criticality 

Rank 

Skill 

Task 

Category 

Sub-Tasks 

Standard 

12 

Transmit  information 

about  the  location  of 

threat  forces,  terrain, 

and  obstacles  that 
influence  operations 

Conduct  Aerial 

Observation 

•  Recognize  threats  to  mission  from  enemy, 
terrain,  weather,  or  other  obstacles . 

•  Report  threats . 

•  Maintain  contact  with  threat  until 

handover  or  told  to  move  on. 

•  Ensure  information  complies  with  Anny 
infomiation  standards. 

•  Infomiation  was  provided  in  the  correct 
format  and  was  complete,  clear,  accurate, 
reliable,  precise,  usable,  and  timely. 

13 

Conduct  target 
handover 

Share  Targeting 
Information 

•  Provide  alert  of  target  handover  and 
description 

•  Establish  time  and  location  of  handover. 

•  Ensure  no  gap  in  target  coverage  occurs 
during  handover. 

•  Handover  target  responsibility  and 
infomiation  about  target  (including  recent 
target  behavior). 

•  After  handover,  maintain  contact  with 
other  party  until  coverage  is  firmly 
established. 

•  Verily  that  party  losing  contact  is  free  to 
depart. 

•  Target  was  not  lost  during  the  handover 
of  tracking  responsibility. 

•  Target  was  acquired  by  the  receiving 
party  in  a  timely  manner. 

14 

Provide  early 
warnings,  ambush 
detection,  overwatch, 

threat  identification 

Conduct  Aerial 

Observation 

•  Reconnoiter  the  route  and  terrain. 

•  Find  all  threats  and  obstacles  that 

influence  movement. 

•  Maintain  enemy  contact. 

•  Report  route  and  threat  infomiation. 

•  Prepare  to  support  friendly  reaction  to 
threats . 

•  Detected  and  reported  enemy  anti-air 
forces  before  they  are  able  to  attack  friendly 
elements . 
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Criticality 

Rank 

Skill 

Task 

Category 

Sub-Tasks 

Standard 

15 

Provide  the  target 
location  (i.e.,  direction 
of  target  in  degrees 
and  range  from  battle 
position) 

Share  Targeting 
Information 

•  Determine  target  location  as  accurately  as 
possible. 

•  Develop  description  of  target  locations  if 
lasing  is  not  reliable  due  to  weather 
conditions,  other  obscurations,  or  range. 

•  Transmit  target  location  relative  to  UA 
location. 

•  Ensure  information  complies  with  Amiy 
information  standards. 

•  Infomiation  was  provided  in  the  correct 
format  and  was  complete,  clear,  accurate, 
reliable,  precise,  usable,  and  timely. 

16 

Provide  confirmation 
of  the  target  prior  to 
engagement 

Identify  Threats 

•  Acquire  the  best  target  identification 
picture  feasible. 

•  Identify  target  as  actionable,  or  not. 

•  Relate  confidence  level  of  target 
identification  to  authorizer  of  fire. 

•  Respond  to  requests  for  more 
information. 

•  Identified  target  as  enemy  prior  to  release 
of  munitions. 

17 

Provide  the  direction  of 
the  target  in  degrees 
and  range  from  the 
battle  position 

Call  for  and 
Adjust  Indirect 
Fires 

•  Determine  target  location  as  accurately  as 
possible. 

•  Develop  description  of  target  locations  if 
lasing  is  not  reliable  due  to  weather 
conditions,  other  obscurations,  or  range. 

•  Transmit  target  location  relative  to  UA 
location. 

•  Ensure  infomiation  complies  with  Army 
information  standards. 

•  Infomiation  was  provided  in  the  correct 
format  and  was  complete,  clear,  accurate, 
reliable,  precise,  usable,  and  timely. 

18 

Transmit  imagery, 
sensor  data,  tactical 
situational  maps, 
overlays,  and  reports 
(e.g.,  spot  reports) 

Conduct  Aerial 

Observation 

•  Receive  and  understand  requirement  to 
transmit  information. 

•  Prepare  infomiation  in  proper  formats. 

•  Disseminate/transfer  infomiation  over 
proper  communications  channels. 

•  Ensure  infomiation  complies  with  Amiy 
information  standards. 

•  Infomiation  was  provided  in  the  correct 
format  and  was  complete,  clear,  accurate, 
reliable,  precise,  usable,  and  timely. 
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Criticality 

Rank 

Skill 

Task 

Category 

Sub-Tasks 

Standard 

19 

Provide  accurate 
description  of  target  to 
support  target 

selection 

Identify  Threats 

•  Gain  all  infonnation  required  for  target 
selection  standard. 

•  Prepare  target  report. 

•  Transmit  target  report. 

•  Respond  to  requests  for  more 
infonnation. 

•  Prepare  to  support  friendly  reaction  to 
target  report. 

Target  infonnation  contained  all  the  elements 
required  to  support  target  selection: 

•  Type 

•  Identification 

•  Activity 

•  Number 

•  Location 

20 

Utilize  standardized 

radio  communication 
and  signal  operating 
procedures 

Communicate 

Requirements 

•  Utilize  radio  tenninology  ffomDoD  Flight 
Infonnation  Publication. 

•  Utilize  equipment  tenninology  from 
operator’s  manual. 

•  Restrict  official  transmissions  to 
established  communications  procedures  and 
tenninology. 

•  Police  the  net  when  variances  are 

detected. 

•  Communication  was  IAW  tactical 
communications  SOP  and  Army  aviation 
doctrine. 

•  Communication  did  not  degrade  or  delay 
unit  operations 

21 

Perform  battle  damage 
assessment 

Provide  BDA 

•  Collect  required  infonnation  for  BDA 
reporting. 

•  Complete  BDA  report. 

•  Send  BDA  report,  and  respond  to 
requests  for  more  information. 

•  Ensure  information  complies  with  Army 
infonnation  standards. 

•  BDA  report  IAW  TACSOP  and  FM  101-5- 
2,  and  was  clear,  accurate,  reliable,  precise, 
usable,  and  timely. 

22 

Find/track  targets 

Identify  Threats 

•  Recognize  target  as  a  high  priority  target. 

•  Report  HPT  IAW  TACSOP. 

•  Maintain  contact  with  threat  until 

handover  or  told  to  move  on. 

•  Prepare  to  support  friendly  reaction  to 

HPT  report. 

•  Prioritized  targets  were  identified  and 
reported,  and  UAS  did  not  lose  contact  with 
target  before  handover  or  mission  completion. 

Criticality 

Rank 

Skill 

Task 

Category 

Sub-Tasks 

Standard 

23 

Transmit  information 

about  the  unit  of  action 

Conduct 

Cooperative 

Engagements 

•  Determine  UA  location  relative  to  target 
location  as  accurately  as  possible. 

•  Develop  description  ofUA  relative  to  the 
target,  with  more  detail  if  the  UA  is  not 
showing  on  the  aviation  unit  COP. 

•  Transmit  UA  location  relative  to  target 
location. 

•  Verify  aviation  understanding  of  UA 
location. 

•  Ensure  information  complies  with  Amiy 
information  standards. 

•  Aviation  unit  aircraft  were  able  to 

determine  UA  location  even  if  not  shown  on 

their  COP. 

•  Information  was  provided  in  the  correct 
format  and  was  complete,  clear,  accurate, 
reliable,  precise,  usable,  and  timely. 

24 

Utilize  standardized 
report  formats 

Communicate 

Requirements 

•  Maintain  current  references  for  reports 
from  doctrine  or  TACSOP. 

•  Select  appropriate  report  format  for  the 
type  of  information  to  be  delivered. 

•  Determine  proper  communications 
method  to  transfer  reports. 

•  Communication  was  in  accordance  with 
tactical  communications  SOP  and  Amiy 
aviation  doctrine. 

25 

Provide  target 

description 

information 

Share  Targeting 
Information 

•  Collect  all  required  information  on  target. 

•  Complete  spot  report  with  information 
required  for  target  type,  specific  information 
requirements,  purpose  of  report,  etc. 

•  Transmit  report  IAW  TACSOP. 

•  Ensure  information  complies  with  Amiy 
information  standards. 

•  Target  description  contained  all  required 
elements  for  direct  or  indirect  attack,  to 
include  for  the  type  of  munitions  to  be  fired. 

Notes:  TACSOP  =  tactical  standard  operating  procedure;  IAW  =  in  accodance  with;  FRA  GO  =  framentary  order;  AO  =  area  of  operation; 
COP  =  common  operating  picture;  SOP  =  standard  operating  procedure;  BDA  =  battle  damage  assessment;  HPT  =  high  priority  target;  UA  = 
unit  of  action. 


APPENDIX  F 

BENCHMARK  INDICATORS,  CONDITIONS,  AND  SOURCES  FOR 
MANNED-UNMANNED  TEAMING  SKILLS 


F-l 


Indicators,  Conditions,  and  Sources  for  MUM-T  Skills  Benchmarks 
(Presented  in  training-criticality  order) 


1.  Deconflict  munition  trajectories  from  airframes 

Indicators  Call  for  fire  is  complete  and  accurate 

Operator  is  aware  that  deconfliction  is  taking  place 
Coordinates  with  ATC  for  deconfliction 

Identifies  intersecting  munition  trajectories  and  recommends  lateral,  vertical,  or  sequential  maneuver 

Operator  is  aware  of  positions  of  friendly  aviation  assets 

Responds  to  instructions  to  deconflict.  Moves  aircraft  to  safe  area 

Confirms  when  clear 

All  aircraft  are  in  assigned  locations 

Air  Mission  Commander  has  to  redirect  the  UAS  from  munition  trajectory  or  impact  area 
Air  Mission  Commander  has  to  call  a  cease  fire  due  to  UAS  location 
Determines  if  rounds  are  accurately  placed  on  target 
Appropriate  graphics  are  displayed  and  current 
Conditions  Call  for  Fire  missions;  own  or  other  unit 
Troop  suppression 
Show  of  force 

Preplanned  coordinated  attack 
Target  of  opportunity  with  priority  of  fires  assumed 
Sources  Virtual  simulations;  AVCATT,  PIMS 
Live  simulations 
Gunnery  training 
Combat 

2.  Utilize  standardized  execution  commands  to  initiate  attack 

Indicators  SPOT  reports  are  complete,  accurate,  and  in  correct  format 
Determines  target  location  and  relays  appropriately 
Uses  proper  radio  procedures  IAW  TACSOP 
Terminology  and  sequence  are  IAW  current  JFIRE  publications 
Understands  commands,  expected  responses  and  order  of  steps 
Uses  internal  Position  feedback  communication 
Gets  positive  feedback  (i.e.  closed  loop  communication) 

Fails  to  identify  danger  close  condition 
Fails  to  assess  fratricide  or  collateral  damage 
Fails  to  positively  identify  the  target 
Fails  to  ensure  clearance  of  fires 
Conditions  Call  for  Fire  missions;  own  or  other  unit 

Unit  in  contact  teaming  with  attack  platforms,  mortar  or  FA  teams 
On  identification  of  a  target  for  engagement 
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On  receipt  of  a  request  for  targeting  information 


Sources  Virtual  simulations;  AVCATT,  P1MS 

Live  simulations 
Gunnery  training 
Combat 

Table  talk  discussion 
Garrison  training 

Operation  within  an  ATC  command  airspace 

3.  Transmit  information  about  the  method  of  attack 

Indicators  Transmits  appropriate  information 

Uses  secure  net  properly 
Uses  JFIRE  manual 

Updates  locations  of  friendlies  and  enemy  targets 
Chooses  appropriate  scheme  of  maneuver 

Understands  formations  for  mounted/unmounted  maneuver  (e.g.  wedge,  bounding  overwatch) 
Identifies  defensive  postures  and  positions 
Identifies  fire  support  elements  and  likely  locations 
Understands  distribution  of  fires 

Follows  the  appropriate  checklist  for  the  munition  selected 
Provides  verbal  feedback  &  acknowledgments 
Conditions  Call  for  Fire  missions;  own  or  other  unit 

Unit  in  a  defensive  position  and  under  attack 
Offensive  operations 
MOUT  operations 

Sources  Virtual  simulations;  AVCATT,  PIMS 
Live  simulations 
MOUT  sites 

National  Training  Center 
Combat 

Table  talk  discussion 

4.  Switch  roles  of  laser  designator  and  missile  launch  platforms 

Indicators  Understands  both  designator  and  shooter  roles 
Knows  weapons  limitations  and  effects 
Communicates  transfer  of  responsibilities  and  confirms 
Transmits  target  location,  description,  laser  code,  laser  target  line 
Uses  standard  execution  commands 
Clears  area  of  fire,  UA  and  friendly  ground  forces 
Repositions  airframe  correctly 
Confirms  LASER  code  is  same  as  munitions  code 
Understands  LASER  designator  operation 
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Follows  launch  procedures 
Receipt  of  transfer  request 
Attack  mission 

FA  teaming  if  possible  beyond  Line  of  Sight  firing 
Quick  reaction  to  convoy  attack 
Periodic  currency  requirements  training 

Conditions 

Sources  Virtual  simulations;  AVCATT,  PIMS 

Live  simulations 
CAB  training  event 
Convoy  operations 
Table  talk  discussion 
Combat 

Gunnery  training 

5.  Conduct  call  for  direct  fires 

Indicators  Employs  correct  communication  channels 

Executes  standard  Call  for  Fire  commands  and  makes  correct  responses 
Follows  TACSOP 
Complies  with  ROE 

Understands  both  designator  and  shooter  roles 
Knows  scheme  of  maneuver 
Confirms  locations  of  friendlies 
Uses  checklists 

Meets  Safety  and  employment  guidelines 
Knows  correct  clearance  of  fire  authority 
Uses  risk  mitigation  procedures 

Knows  the  munitions  effects  (expected  trajectories,  burst  radius,  etc)  and  safe  clearances 
Clears  area  of  fire,  UA  and  friendly  ground  forces 
Observes  target  effects 
Conditions  On  request  for  direct  fires 
FA  teaming  (live  fire) 

Infantry  mortar  platoons  teaming 

Sources 

Virtual  simulations;  AVCATT,  PIMS 

Live  simulations 

Combat 

Classroom 

Discussion  with  manned  element  (briefing  area) 

Gunnery  training 
Counter  Insurgency  training 


F-4 


6.  Select  the  best  weapon  systems  to  engage  the  target 
Indicators  Identifies  target,  enemy  locations  and  friendly  locations 

Interprets  commander's  intent 

Understands  ROE 

Understands  scheme  of  maneuver 

Understands  weapons  characteristics  limitations 

Understands  munitions  and  effects 

Correctly  matches  platforms  with  weapon  systems 

Correctly  discriminates  hardened  sites  and  soft  targets 

Performs  BDA 

Evidence  of  undesired  effect  on  target 
Excessive  collateral  damage 
Anticipates  re-engagement 
Conditions  During  engagement  scenarios 
Tank  or  FA  live  fire 
Mortar  live  fire 

Sources  Virtual  simulations;  AVCATT,  P1MS 

Live  simulations 
Combat 
Table 
Combat 

Gunnery  training 

7.  Develop  and  send  COP  information  to  air-ground  team 

Indicators  Exhibits  loss  of  situational  awareness 

Determines  if  new  information  coincides  with  or  diverges  from  Air  Mission  Brief 

Relays  information  to  all  parties  involved 

Understands  preceding  COP 

Understands  commander's  intent 

Updates  situation  frequently 

Transmits  information  IAW  TACSOP  and  checklists 
Conditions  Change  in  scenario  or  situation 

Sources  Virtual  simulations;  AVCATT,  PIMS 

Live  simulations 
Combat 

Field  Training  Exercise 
National  Training  Center 

8.  Utilize  joint,  Army,  and  civilian  personnel  recovery  terminology 

Indicators  Understands  COP  in  relation  to  PR  assets  in  play 
Provides  accurate  SPOT  reports 
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Reporting  is  IAW  FM  3-04,  126 
Conditions  Flas  information  about  PR  for  transmission 
Receives  information  about  PR 
Sources  Virtual  simulations;  AVCATT,  P1MS 

Live  simulations 
Combat 

Field  Training  Exercise 

9.  Prioritize  the  engagement  of  targets 

Indicators  Understands  target  priority  list 

Understands  commander's  intent 
Prioritization  is  IAW  commander's  intent 
Understands  friendly  scheme  of  maneuver 
Knows  threat  systems  and  capabilities 

Knows  reasons  for  prioritization  of  targets  (e.g.  wind/smoke  obscuration) 
Understands  ROE 

Conditions  Encounter  with  threat 

Sources  Virtual  simulations;  AVCATT,  PIMS 

Live  simulations 
Combat 

Field  Training  Exercise 

10.  Gain  and  Maintain  Enemy  Contact 

Indicators  Updates  COP  promptly 

Provides  early  and  accurate  threat  warning 
Communicates  contact  information 
Informs  when  contact  is  lost 

Demonstrates  Area  of  Operations  and  Plan  of  Operations  expertise 
Uses  positive  target  identification  procedure 
Able  to  identify  the  target,  observe  what  target  is  doing 
Anticipates  cover,  concealment  and  terrain  masking 
Assesses  terrain  for  most  likely  route  of  target 
Conditions  Reconnaissance  missions 

Virtual  simulations;  AVCATT,  PIMS 

Live  simulations 

Combat 

Field  Training  Exercise 

Sources 

11.  Conduct  call  for  indirect  fires 

Indicators  Acknowledges  fire  mission  receipt 

Employs  closed  loop  radio  procedure 

Uses  terminology  and  sequence  IAW  JFIRE  and  FM  6-30,  IAW  TC  1-248 
_ Coordinates  with  supporting  Fire  Support  Element _ 
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Knows  clearance  authority 
Sources  Virtual  simulations;  AVCATT,  PIMS 
Live  simulations 
Gunnery 

Field  Training  Exercise 

12.  Transmit  information  about  the  location  of  threat  forces,  terrain,  and  obstacles  that  influence 
operations 

Indicators  Transmits  SPOT  reports  in  SALT-W  format  1AW  TC  1-248 
Identifies  imminent  threats  to  the  manned  element 
Identifies  obstacles  to  maneuver 
Knows  current  Priority  Intelligence  Requirement 
Knows  terrain  effects  on  helicopter  maneuvering 
Conditions  On  redirection  of  helicopter  element 
Sources  Virtual  simulations;  AVCATT,  PIMS 
Live  simulations 
Gunnery 

Field  Training  Exercise 

13.  Conduct  target  handover 

Indicators  Proceeds  IAW  TC  1-248 

Uses  8-line  message  to  redirect,  then  hand  over  to  helicopter 
Correctly  reads  grid  locations 

Identifies  out  of  the  ordinary  behavior  or  placement  of  vehicles 
Maintains  awareness  of  relative  positions  and  obstacles 

Ensures  target  designator  energy  is  not  blocked  from  helicopter  element  by  terrain  or  obstacles 
Stays  on  target  until  handover  is  complete 
Conditions  Target  handover  or  relief  on  station 
Sources  Virtual  simulations;  AVCATT,  PIMS 

Live  simulations 

14.  Provide  early  warnings,  ambush  detection,  overwatch,  threat  identification 

Indicators  Identifies  out  of  the  ordinary  behavior  or  placement  of  vehicles 

Transmits  information  sent  to  the  appropriate  recipient  IAW  Air  Mission  Brief 
Interprets  Air  Mission  Brief  for  the  UAS  role 
Knows  what  a  threat  (e.g.  ambush)  looks  like 
Conditions  Mission  analysis 

Encounter  with  threat,  ambush  situation 
Sources  Virtual  simulations;  AVCATT,  PIMS 

Live  simulations 
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15.  Provide  the  target  location 

Indicators  Positively  identifies  and  locates  targets 

Sends  grid  and  target  identification  are  sent  to  the  manned  system 
Provides  direction  relative  to  receiver's  position 
Knows  manned  asset  location 
Solicits  manned  asset  location 
Conditions  Target  detection 
Sources  Virtual  simulations;  AVCATT,  PIMS 
Live  simulations 

16.  Provide  confirmation  of  the  target  prior  to  engagement 

Indicators  Uses  appropriate  method  (Spot  tracker,  LASER,  talk-on)  to  get  manned  system  onto  target 
Understands  ROE 

Understands  differences  among  Detect,  Recognize,  and  Identify  a  target 
Uses  positive  target  identification  and  local  SOP 
Knows  commander's  intent 
Conditions  Target  detection 
Sources  Virtual  simulations;  AVCATT,  PIMS 

Live  simulations 

18.  Transmit  imagery,  sensor  data,  tactical  situational  maps,  overlays,  and  reports 
Indicators  Indicators  are  covered  in  the  subtasks 

Understands  the  drivers  of  the  requirements 
Uses  local  SOP 

Knows  current  network  configuration 
Conditions  SR  mission 

Sources  Virtual  simulations;  AVCATT,  PIMS 

Live  simulations 

19.  Provide  accurate  description  of  target  to  support  target  selection 

Indicators  Complies  with  priority  of  fires 

Uses  positive  target  identificaiton  and  local  SOP 
Recognizes  combat  vehicle  systems 
Conditions  Target  detection 
Sources  Virtual  simulations;  AVCATT,  PIMS 

Live  simulations 

20.  Utilize  standardized  radio  communication  and  signal  operating  procedures 

Indicators  Knows  application  of  plain  v  cipher  text 
Follows  SOP 
Conditions  SR  mission 

Sources  Virtual  simulations;  AVCATT,  PIMS 

Live  simulations 
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Notes :  ATC  =  air  traffic  control;  AVCATT  =  Aviation  Combined  Arms  Tactical  Trainer;  BDA  =  battle 
damage  assessment;  CAB  =  combat  aviation  brigade;  COP  =  common  operating  picture;  FA  =  field 
artillery;  FM  =  Field  Mannual;  IAW  =  in  accordance  with;  JFIRE  =  joint  application  of  firepower; 
MOUT  =  military  operations  in  urban  terrain;  PIMS  =  Portable  Institutional  Mission  Simulator;  PR  = 
personnel  recovery;  SALT-W=  size,  activity,  location,  time,  and  what  you  wil  do;  SR  =  scout- 
reconnaissance;  TC  =  Training  Circular;  UA  =  unit  of  action. 
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APPENDIX  G 

COMMENTS  BY  SUBJECT  MATTER  EXPERTS  ON  FIVE 
MANNED-UNMANNED  TEAMING  SKILLS 
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Manned-Unmanned  Teaming  Skills  Rank  Ordered  on  Training  Criticality  Index 

1.  Deconflict  munition  trajectories  from  airframes 

Past  experience  has  been  that  all  communications  were  relayed  from  the  GCS  to  the  manned  element  unit 
tactical  operations  center  and  then  to  manned  element  operators. 

Mission  planning  by  the  team  is  not  currently  practiced.  It  may  be  written  in  the  TACSOP.  There  are  essentially 
no  preplanned  MUM-T  missions 

The  disparity  in  mission  endurance  between  manned  (~2  hrs)  and  unmanned  (6-8  hrs)  systems  requires 
accommodation  in  mission  planning  and  execution.  The  UAS  may  team  with  three  different  helicopter  missions 
in  one  flight. 

Manned  and  unmanned  components  presently  have  different  TACSOPs. 

3.Transmit  information  about  method  of  attack 

Failure  to  properly  position  the  UAS  may  prevent  the  manned  platform  from  engaging  a  target. 

All  units  participating  will  require  common  standard  operating  procedure/policies  for  proper  coordination. 

Checklists  for  munitions  should  be  present  in  hard-copy.  It  would  be  better  to  put  on  system  displays. 

Fragmentary  Orders  arrive  in  the  GCS  by  chat,  e-mail,  or  voice. 

7.  Develop  and  send  Common  Operating  Picture  information  to  air-ground  team. 

Introduction  of  enhancements  to  the  UAS  mission  package  (e.g.  Synthetic  Aperture  Radar)  will  have  a  large 
impact  on  this. 

18.  Transmit  imagery,  sensor  data,  tactical  situational  maps,  overlays,  and  reports 

The  MUM-T  team  needs  complete  digital  communications 

The  MUM-T  team  elements  need  Command,  Control,  Communications,  Computers,  and  Intelligence  in  order  to 
properly  use  Force  XXI  Battle  Command  Brigade  and  Below,  Blue  Force  Tracker,  Advanced  Field  Artillery 
Tactical  Data  System,  and  other  tools. 

19.  Provide  accurate  description  of  target  to  support  target  selection 

Recognition  of  Combat  Vehicles  course  should  be  an  annual  requirement. 

Notes'.  GCS  —  Ground  Control  Station;  MUM-T  =  manned-unmanned  teaming;  TACSOP  =  tactical  standard 
operating  procedure. 
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